View Logs (Loki)

You can view all the logs related to your deployed inference service through the Loki data source.

How to View Logs #

Navigate to the Deployments dashboard.
Select the Ellipsis icon for the deployment you want to monitor.
Select the Dashboard option. A new browser tab opens with the Grafana dashboard for the selected deployment. By default, all packaged model versions are displayed.
From the Grafana dashboard, navigate to Explore.
Choose the Loki data source.

In the Select label dropdown, select one of the following Label names:

Label Name	Value	Description
`serving.kserve.io/inferenceservice`	The deployment name.	Selects all instances of all versions of your inference service. Selectable in the Deployment Dashboard via the `Deployment Name` dropdown.
`inference/packaged-model`	The packaged model name and version. For example: `fb125m-model.v1`	Selectable in the Deployment Dashboard via the `Packaged Model Version` dropdown. By default, all versions of the deployment are shown.
`inference/deployment-id`	The deployment’s `id` value.	For advanced use. Normally `serving.kserve.io/inferenceservice` is used as long as deployment names are not reused for different instances.
`inference/packaged-model-id`	The packaged model’s `id` value.	For advanced use. Normally `inference/packaged-model` is used as long as packaged model names are not reused for different instances.

In the Select value dropdown, select the corresponding value that matches your deployment or packaged model.

Continue building your query as needed. You can also optinally add container label filters:

Container Type	Description
`kserve-container`	Logs specifically from the running inference service code (bento)
`aioli-logger`	Request/response logs events for the inference service
`queue-proxy`	Internal kserve component supporting scale up/down
`agent`	Internal kserve component tracking requests

Click Run query to view the logs.

Surface the Loki API on port 3100 of the controller.
```
kubectl port-forward svc/loki 3100:3100
```

Send a query to the Loki API.

curl -G -s "http://<YOUR_URL>:3100/loki/api/v1/query_range" --data-urlencode 'query={serving_kserve_io_inferenceservice=~"${NATIVE_DEPLOYMENT_NAME}",container="aioli-logger"}' | jq

View Request/Response Logs #

You can filter the logs to view request/response logs specifically by selecting container="aioli-logger".

{serving_kserve_io_inferenceservice="elb-hf-simple-falcon", container="aioli-logger"}

The request/response logs are in JSON format. You can make them more easily readable by selecting the Grafana Prettify JSON option on the tool bar.

View Stderr/Stdout Logs #

You can filter the logs to view stderr/stdout logs specifically by selecting container="kserve-container".

{serving_kserve_io_inferenceservice="elb-hf-simple-falcon", container="kserve-container"}

How to Change Default Logging Port #

You can change the default logging port by setting the AIOLI_LOGGER_PORT environment variable on your packaged model or deployment. The default port is 49160. This variable works only for custom modelFormat containers.