Set Up a Registry

A registry is the storage location for your models and deployments. Any storage solution that supports the S3 protocol can be used as a registry. Once added, you can use the models to add a deployment for your inference service.

The following table lists the supported model types for each kind of registry you can set up:

Model TypeRegistries
Bento ArchiveS3, PFS
CustomOpenLLM (HuggingFace), PVC, S3, PFS, None
NIMNGC, PVC
OpenLLMOpenLLM (HuggingFace), S3, PFS

Service Deployment Journey

Registry setup is the first step in the service deployment journey. It requires that you have admin access to a compatible storage solution and are familiar with creating and configuring buckets and policy permissions.

graph LR;
    A(Set Up Registry) --> B(Add Registry);
    B --> C(Add Model);
    C --> D(Create Deployment);
    style A fill:#7FF9E2,stroke:#333,stroke-width:4px;
    style B fill:#fff,stroke:#333,stroke-width:2px;
    style C fill:#fff,stroke:#333,stroke-width:2px;
    style D fill:#fff,stroke:#333,stroke-width:2px;