Install & Deploy Controller (Helm)

Before You Start #

Complete the HPE MSC Docker Registry setup
Create secrets for:
- Private Docker Registry (to pull private images)
- TLS (for HTTPS access to API)
Ensure the availability of prerequisites for HPE Machine Learning Inferencing Software installation:
- Kubernetes cluster ≥ 1.20 with Docker
- Helm ≥ 3.0
- KServe ≥ 0.11 (Serverless with its dependencies Istio >= 1.18, Knative >= 1.10, and Cert Manager >= 1.9.0)

Note

MLIS has been installed successfully on:

On-premise:
- Kubernetes
- SLES 15 Rancher (RKE2)
MicroK8s
GKE
Kind
Docker Desktop Kubernetes

EKS is known to not work out-of-the-box due to a need for custom service accounts. You may need to regenerate the service accounts with the required annotations to deploy successfully on EKS.

How to Deploy the Platform #

1. Obtain the Helm Chart #

Log in to the HPE MSC Portal.
Navigate to the Manage section and select the related record for the software version you’ve purchased.
Select the Go to Download Page button.
Agree to any terms if applicable.
Select the items you wish to download.
Select Download.

Download the Helm chart manually using your HPE MSC account username email and the password token.

helm registry login hub.myenterpriselicense.hpe.com --username <HPE_MSC_EMAIL> --password <TOKEN>
helm pull oci://hub.myenterpriselicense.hpe.com/hpe-mlis/<LOWERCASE_SKU>/aioli

Using this method, you will need to provide the Helm chart to the helm install command once you are ready to deploy the platform on your Kubernetes cluster.

You can also obtain the Helm chart automatically during the Helm install process using your HPE MSC account username email and the password token.

This should not be done until you have completed up to Part 4 and are ready to proceed to Part 5.

helm registry login hub.myenterpriselicense.hpe.com --username <HPE_MSC_EMAIL> --password <TOKEN>
helm install oci://hub.myenterpriselicense.hpe.com/hpe-mlis/<LOWERCASE_SKU>/aioli

Helm Password Token

The password token required for the helm registry login options can be retrieved by clicking the Access Your Products link in the HPE Software Delivery Receipt Email. It is displayed as the key value under the Product Info column.

2. Install KServe #

Refer to the official KServe serverless installation guide for instructions on how to install KServe on your Kubernetes cluster.

3. Configure DNS #

Access to inference service deployments is best achieved by configuring your Kubernetes cluster with a DNS domain name so that your inference services are accessible via a hostname. Otherwise, you must set up port forwarding in Part 6 to access the services (see the Pending IP tab).

Magic DNS Services

If you do not have a DNS domain name for your Kubernetes cluster and do not wish to use port forwarding, you could instead use a magic DNS service such as nip.io to map an external IP to a hostname.

To configure the magic DNS service, create a ConfigMap in the knative-serving namespace with the following command:

kubectl create configmap config-domain \
--namespace knative-serving \
--from-literal <CLUSTER-EXTERNAL-IP>.nip.io="" \
--dry-run=client -o yaml | kubectl replace -f -

This will enable you to access the cluster via the hostname <CLUSTER-EXTERNAL-IP>.nip.io and an inference service via http://<INFERENCE-SERVICE-NAME>/<NAMESPACE>/<CLUSTER-EXTERNAL-IP>.nip.io. See the Configure Magic DNS Service guide for more detailed steps.

Alternative magic DNS solutions include: localtls, sslip.io, and logal.gd.

4. Evaluate Deployment Needs #

Evaluate your deployment needs for the following optional configurations that involve the Helm chart:
- TLS/HTTPS: Enable HTTPS access to the REST API.
- External Auth: Enable external authentication services using OIDC and other identity providers.
- Observability: Disable or enable observability features.
- Node Selectors: Control which nodes HPE Machine Learning Inferencing Software is deployed on.
- Rancher Engine: Configure Loki Service to work with CoreDNS if applicable.
- Namespace In/Exclusions: Filter out namespaces not to be used for deployment.
- GPU Selection: Enable selecting GPU types when defining a packaged model.
Optionally update your values.yaml file with the necessary configurations for any of the previous options.
Add it to the helm install mlis command in the next step using --values values.yaml.

Configure Garbage Collection #

Unless otherwise configured, inactive inference service replicas shut down after 15 hours. Knative provides Garbage collection configuration options to customize the cleanup behavior of inactive replicas. GPUs allocated to a replica are not released when the replica is shut down.

We recommend that you configure Knative for immediate cleanup of inactive replicas (Knative Revisions). The following command will configure Knative to immediately clean up inactive replicas:

kubectl patch cm  -n knative-serving config-gc  --type=strategic \
      -p '{"data":{"min-non-active-revisions":"0", "max-non-active-revisions": "0", "retain-since-create-time": "disabled","retain-since-last-active-time": "disabled"}}'

5. Install the Controller via Helm #

Download the Helm chart from your HPE MSC account.
Run the helm install command, passing in:
- Your required secrets
- An optional values.yaml file for configurations (TLS, external auth)
- A default password for the admin user

The following command assumes you have obtained the Helm chart by downloading it manually, either via the HPE MSC Portal or the helm pull method.

helm install mlis \
   --set 'global.imagePullSecrets[0].name=regcred' \
   --set 'global.imagePullSecrets[1].name=hpe-mlis-registry' \
   --set imageRegistry=hub.myenterpriselicense.hpe.com/hpe-mlis/<LOWERCASE_SKU> \
   --set defaultPassword=<CREATE_ADMIN_PASSWORD> \
   --values values.yaml \ # Required if you have custom configurations
   <SKU>_aioli-helm-chart<release/majorMinorPatchNumber>}}.tgz

The following command assumes you have performed the helm registry login as described in Part 1 and are ready to install the platform.

helm install mlis \
   --set 'global.imagePullSecrets[0].name=regcred' \
   --set 'global.imagePullSecrets[1].name=hpe-mlis-registry' \
   --set imageRegistry=hub.myenterpriselicense.hpe.com/hpe-mlis/<LOWERCASE_SKU> \
   --set defaultPassword=<CREATE_ADMIN_PASSWORD> \
   --values values.yaml \ # Required if you have custom configurations
   oci://hub.myenterpriselicense.hpe.com/hpe-mlis/<LOWERCASE_SKU>/aioli

Obtain Default Admin Password if Not Set

If you skip setting the admin password during installation (using --set defaultPassword), the system generates one automatically. You can retrieve this generated password by using the following command and replacing <RELEASE_NAME> with the name of the Helm release (e.g., mlis):

   kubectl get secrets aioli-master-config-<RELEASE_NAME> \
   --template='{{ index .data "aioli-master.yaml" | base64decode }}' | grep defaultPassword

If you use this command after updating the admin password, it will still only return the original default password.

6. Obtain Controller Address #

Once installed, identify the host/ip to be used to communicate with the Controller. Typically this is via the EXTERNAL-IP of the aioli-proxy service.

Enter the following command to obtain the external IP address of the aioli-proxy service.

kubectl get svc/aioli-proxy

NAME          TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)        AGE
aioli-proxy   LoadBalancer   10.80.7.201   35.224.128.245   80:30819/TCP   17m

If an IP address is assigned, you can obtain the controller address with the following command. The Helm chart NOTES provide the exact command based upon any customization you may have made.

 export AIOLI_CONTROLLER="http://$(kubectl get svc aioli-master-service-aioli --output jsonpath='{.status.loadBalancer.ingress[0].ip}'):80"

If your Kubernetes cluster is not configured to provide an external address, the EXTERNAL-IP will remain <pending>. You can use a port-forward via the ingress gateway service with the endpoint shown in the deployment state using the following commands:

export INGRESS_HOST=localhost
export INGRESS_PORT=8100
INGRESS_GATEWAY_SERVICE=$(kubectl get svc --namespace istio-system --selector="app=istio-ingressgateway" --output jsonpath='{.items[0].metadata.name}')
kubectl port-forward --namespace istio-system svc/${INGRESS_GATEWAY_SERVICE} ${INGRESS_PORT}:80
export AIOLI_CONTROLLER="http://${INGRESS_HOST}:${INGRESS_PORT}:80"

Once you’ve identified how to access the HPE Machine Learning Inferencing Software controller, setup the AIOLI_CONTROLLER export variable to reference the proper address by default or specify it via the AIOLI command option -c/--controller.

CLI TLS/HTTPS SUPPORT

If authentication has been enabled, the AIOLI_CONTROLLER address should be defined to use https://....

The CLI can be configured with environment variables for AIOLI_CONTROLLER_CERT_FILE and AIOLI_CONTROLLER_CERT_NAME to point to full certificate chain. If the certificate is not signed by a well-known Certificate Authority (CA), the user can adopt a trust-on-first-use approach by trusting the SHA256 fingerprint of the certificate chain.