From Registry (UI)

Before You Start #

• Set up a registry
• Confirm that your model is available in your chosen registry

Basic Details #

1. Sign in to HPE Machine Learning Inferencing Software.
2. Navigate to Packaged Models.
3. Select Add new model.
4. Input details for the following:
   • Name: The name of the model within HPE Machine Learning Inferencing Software.
   • Description: A brief description of the model.
5. Select Next.

Storage Details #

1. Input details for the following:
   • Registry: The registry where the model is stored.
   • Model Format: Options are OpenLLM, Bento archive, NIM, Custom.
   • Image: The container image servicing the model; must be the name of the image + a release tag. For NIM, see the NGC catalog for the image options.
   • URL/Path: The URL to the model object in the registry. (e.g., s3://<BUCKET_NAME>/models/<MODEL_NAME>.tar.gz)

     Prefix	Description
     openllm://	An openllm model name from huggingface.co dynamically loaded and executed with a VLLM backend.
     s3://	An openllm model path which will be dynamically downloaded from an associated s3 registry bucket and executed with a VLLM backend.
     pvc://	A PVC model path that can be used for pre-downloaded NIM and Custom models.

2. Select Next.

Resource Templates #

1. Choose a Resource Template or define custom resources.

   Name	Description	Request CPU	Request Memory	Request GPU	Limit CPU	Limit Memory	Limit GPU
   cpu-tiny	1 cpu, 10Gi memory, no gpu per replica	1	10Gi		1	10Gi
   cpu-small	4 cpu, 20Gi memory, no gpu per replica	4	20Gi		6	40Gi
   cpu-large	8 cpu, 40Gi memory, no gpu per replica	8	40Gi		10	60Gi
   gpu-tiny	1 cpu, 10Gi, 1 gpu per replica	1	10Gi	1	1	10Gi	1
   gpu-small	2 cpu, 20Gi, 2 gpu per replica	2	20Gi	2	6	40Gi	2
   gpu-large	8 cpu, 40Gi, 4 gpu per replica	8	40Gi	4	10	60Gi	4

   GPU Type

   Specifying a GPU type requires heterogenous GPU support be enabled.

2. Select Next.

Environment Variables & Arguments #

Environment variables and arguments are advanced configuration options that you can set for your packaged model. These inputs will vary based on your model’s requirements. For more information, see the Advanced Configuration reference article.

1. Provide any needed Environment Variables.
2. Provide any needed Arguments.
3. Select Create model.