From Registry (CLI)

Before You Start #

Set up a registry
Confirm that your model is available in your chosen registry
OpenLLM: Sign up for a Hugging Face account and create an access token.
- Profile > Settings > Access Tokens
- New Token
NGC: Sign up for an Nvidia NGC account and obtain the necessary API key.
- Profile > Setup > Generate API Key

Supported Registry URI Syntax #

Review the supported registry URI syntax before adding a model from a registry. This URI is passed in using the --url flag.

Prefix	URI Syntax	Description
openllm://	`openllm://<model-ref>`	An openllm model name from huggingface.co dynamically loaded and executed with a VLLM backend.
s3://	`s3://<bucket-name>/<path-to-model>`	A model directory dynamically downloaded from an associated s3 registry bucket. This is supported for the bento-archive, openllm, and custom model formats.
pvc://	See From PVC setup guides	A PVC model path that can be used for pre-downloaded NIM and Custom models.
pfs://	`pfs://<project>/<repo>@<commit>[:<path>][?containerPath=<path>]`	A PFS model path that can be used for models stored in HPE Machine Learning Data Management repositories.
ngc://	Not supported

How to Add a Packaged Model From a Registry #

Use the following command to sign in:

   aioli user login <YOUR_USERNAME>

Step 2: List Available Models and Obtain Values #

Retrieve the allowed image and URL values for your model with the following command:

aioli registries models <REGISTRY-NAME>

The output of this command provides:

URLs for openllm and PFS registries.
Image specifications for NGC registries (URLs are not used for nim-format packaged models).

Step 3: Create a New Model #

Now, create a model using the retrieved values:

   aioli model create <MODEL_NAME> \
   --description <DESCRIPTION> \
   --image <USER_NAME>/<MODEL_NAME>:<TAG> \
   --url <OBJECT_URL> \
   --registry <REGISTRY_NAME> \ 
   --enable-caching

For more information on the aioli model create command, see the CLI command reference.
For information on setting environment variables and arguments, see the Advanced Configuration reference article.