Developer System Setup

The following steps guide you through setting up your developer system to create and deploy containerized inference services. The HPE Machine Learning Inferencing Software developer environment is based on a Python (version 3.9 or greater) environment and includes commands from the CLI, BentoML, and OpenLLM.

info icon Info
You only need to complete this setup if you intend to use the CLI.

How to Set Up Your Developer System

warning icon Must Install in Correct Order

You must install the dependencies separately and in the order listed below.

If you try to install them at the same time (pip install bentoml==x openllm==x aioli-sdk==x), you may run into compatibility errors.

  1. Create a virtual environment to allow HPE Machine Learning Inferencing Software to have its own Python dependencies and configuration, without interfering with other projects or the system-wide Python installation. 
    python -m venv ~/.virtualenvs/mlis
  2. Activate the virtual environment to make it the active Python environment for your current shell session. 
    source ~/.virtualenvs/mlis/bin/activate
  3. Install the latest version of pip. 
    python -m pip install --upgrade pip
  4. Install BentomL and OpenLLM. 
    python -m pip install bentoml==1.1.11 openllm==0.4.44 openllm-client==0.4.44
  5. If you have a supported GPU (Nvidia), install the OpenLLM VLLM backend. This includes the appropriate drivers needed to build an inference service container that supports GPUs. 
    python -m pip install "openllm[vllm]"
  6. Install the SDK. 
    python -m pip install aioli-sdk==1.2.0
  7. Install the AWS CLI and configure it with your AWS credentials. You may need to create an IAM user with the necessary permissions and pass in the access key and secret key through the AWS CLI. 
    brew install awscli
  8. Install any other python dependencies your inference service may need into the same python environment.

Next Steps

You are now ready to either: