Setting up a model type in Criterion AI

After having created the first version of your custom model, wrapped it into a Python package and created your settings schema, you can set up a new model type in Criterion AI to test out your model. You must be an administrator of your organization in order to create a model type so make sure you have this role before continuing. Click on your username in the top right-hand corner and click on Overview under Model types to open the list of model types specific to your organization.

Click on Create model type to create a new model type.

Fill out the information in the form to create the model type. See the screenshot velow.

You must give your model type a name and provide values for the following fields:

Variable Explanation
Package path This is the path over Google Cloud Storage's protocol (i.e., the path must start with gs://). You must have your own Google Cloud Platform project and a bucket in Google Cloud Storage to host your package. You can sign up to a free account in Google Cloud Platform if you do not already have one.

Either make your package publicly available or grant reader rights to the package file in your GCS bucket to the following service accounts:
[email protected]

When training on CPUs and GPUs:
[email protected]

When training on TPUs:
[email protected]
Package version This is the version of your package. We suggest going for semantic versioning when choosing a version number for your model type.
Settings schema path This is the path to your settings schema JSON file. Your file must be served over HTTPS (i.e., the path must start with https://). You can host files over HTTPS in Google Cloud Storage so, if you decide to serve your JSON file from there, upload it to your bucket and make it publicly available. If you use Google Cloud Storage, you must also configure Cross-Origin Resource Sharing (CORS) so that the origin https://app.criterion.ai can access your file. You learn more about CORS on Google's documentation site.

You can see an example of what the CORS file for your bucket in Google Cloud Storage could look like below.

[
    {
      "origin": ["https://app.criterion.ai"],
      "responseHeader": ["Content-Type"],
      "method": ["GET"],
      "maxAgeSeconds": 3600
    }
]
		
Python train module This is the name of the train module in your Python package. This value will be fed into the --module-name parameter when initiating the training of models based on your model type. Learn more about this parameter in the article on how to structure your Python package.
Python test module This is the name of the test module in your Python package. This value will be fed into the --module-name parameter when initiating the testing of models based on your model type. Learn more about this parameter in the article on how to structure your Python package.
Python version This is the version of Python that you wish to use. Currently, only 2.7 and 3.5 are acceptable values.
Google Cloud ML Engine runtime version This is the version of the Google Cloud ML Engine runtime that should be used. See Google's Runtime Version List to see valid values for this
Google Cloud ML Engine scale tier This is the scale tier of Google Cloud ML Engine. BASIC just gives you a CPU while BASIC_GPU gives you an NVIDIA Tesla K80 and BASIC_TPU gives you a Tensor Processing Unit, which is Google’s custom-developed ASIC used to accelerate machine-learning workloads. Learn more about how to use GPUs and TPUs, respectively, in your training and testing scripts on Google's documentation site.

Once you have entered the required information, you can click on the button Create to create the model type. By default, your model type will have the status In development, which means that models created using your model type cannot be trained in the cloud. In order to test out your model type while still in development, you will have to run the training locally. Please see the next article to learn how to do so.

Still need help? Contact Us Contact Us