Create Image (Bentoml)

The following steps guide you through creating a containerized image for an inference service using the bentoml CLI. The result of this is a publicly accessible container published at <user>/<model-name> which can be referenced in an HPE Machine Learning Inferencing Software Model definition and deployed as a service.

Before You Start #

• Ensure you have completed Developer System Setup.
• Ensure you have docker installed and running.

Custom Models #

If you have a custom ML model, you must provide some additional configuration to describe the desired inference service interface. See the BentoML Iris Classification Example for reference. Generally, the following three small files need to be provided:

• save_model.py: A script to import and register it in BentoML’s model registry on the local filesystem.
• service.py: Defines the BentoML service, including the API endpoints for model inference.
• bentofile.yaml: Configuration file to build a BentoML service, specifying dependencies and Python environment setup and generated into the container.

How to Create a Containerized Image #

1. Build the container using the bentoml build command.

   bentoml build -f bentofile.yaml

   Copy
2. Containerize the model using the bentoml containerize command.

   bentoml containerize iris:latest --opt platform=linux/amd64  --image-tag iris:latest

   Copy
3. Get the IMAGE ID of the resulting container.

    docker image ls

   Copy
4. Tag the resulting container.

   docker tag <image-id> <user>/<model-name>

   Copy
5. Push the container to a publicly-accessible docker repo.

   docker push <user>/<model-name>

   Copy
6. Verify the container is accessible.

   docker pull <user>/<model-name>

   Copy

About Bento

See the BentoML documentation for details for other options for configuration/building.