Set Up a Registry #
A registry is the storage location for your models and deployments. Any storage solution that supports the S3 protocol can be used as a registry. Once added, you can use the models to add a deployment for your inference service.
The following table lists the supported model types for each kind of registry you can set up:
Model Type | Registries |
---|---|
Bento Archive | S3, PFS |
Custom | OpenLLM (HuggingFace), PVC, S3, PFS, None |
NIM | NGC, PVC |
OpenLLM | OpenLLM (HuggingFace), S3, PFS |
Service Deployment Journey #
Registry setup is the first step in the service deployment journey. It requires that you have admin access to a compatible storage solution and are familiar with creating and configuring buckets and policy permissions.
graph LR; A(Set Up Registry) --> B(Add Registry); B --> C(Add Model); C --> D(Create Deployment); style A fill:#7FF9E2,stroke:#333,stroke-width:4px; style B fill:#fff,stroke:#333,stroke-width:2px; style C fill:#fff,stroke:#333,stroke-width:2px; style D fill:#fff,stroke:#333,stroke-width:2px;