Monitor Services #
HPE Machine Learning Inferencing Software automatically configures monitoring of all deployed AI inference services. The types of information collected are:
Managing Observability
Inference Service Labels #
Your inference service can be identified via one of its labels.
| Label Name | Value | Description |
|---|---|---|
serving.kserve.io/inferenceservice | The deployment name. | Selects all instances of all versions of your inference service. Selectable in the Deployment Dashboard via the Deployment Name dropdown. |
inference/packaged-model | The packaged model name and version. For example: fb125m-model.v1 | Selectable in the Deployment Dashboard via the Packaged Model Version dropdown. By default, all versions of the deployment are shown. |
inference/deployment-id | The deployment’s id value. | For advanced use. Normally serving.kserve.io/inferenceservice is used as long as deployment names are not reused for different instances. |
inference/packaged-model-id | The packaged model’s id value. | For advanced use. Normally inference/packaged-model is used as long as packaged model names are not reused for different instances. |
Label Names in Grafana
/), hyphens (-), and periods (.), for example, inference/packaged-model becomes inference_packaged_model.Log Container Filters #
You can filter logs by container type when accessing the Loki data source.
| Container Type | Description |
|---|---|
kserve-container | Logs specifically from the running inference service code (bento) |
aioli-logger | Request/response logs events for the inference service |
queue-proxy | Internal kserve component supporting scale up/down |
agent | Internal kserve component tracking requests |
Filtering Containers #
Filter-out a container such as queue-proxy with the option container!="queue-proxy".
Viewing a Specific Container #
View only a specific container such as your image output with container="kserve-container".