Set Up a Registry

A registry is the storage location for your models and deployments. Any storage solution that supports the S3 protocol can be used as a registry. Once added, you can use the models to add a deployment for your inference service.

The following table lists the supported model types for each kind of registry you can set up:

Model Type Registries
Bento Archive S3
Custom OpenLLM (HuggingFace), PVC, S3, None
NIM NGC, PVC
OpenLLM OpenLLM (HuggingFace), S3

Service Deployment Journey

Registry setup is the first step in the service deployment journey. It requires that you have admin access to a compatible storage solution and are familiar with creating and configuring buckets and policy permissions.

graph LR;
    A(Set Up Registry) --> B(Add Registry);
    B --> C(Add Model);
    C --> D(Create Deployment);
    style A fill:#7FF9E2,stroke:#333,stroke-width:4px;
    style B fill:#fff,stroke:#333,stroke-width:2px;
    style C fill:#fff,stroke:#333,stroke-width:2px;
    style D fill:#fff,stroke:#333,stroke-width:2px;