Observability guide

Observability with InspectBoard

InspectBoard is the LLMInspect observability dashboard. Use it to monitor usage and activity across your LLMInspect deployment, including InspectChat conversations, GuardRail validation outcomes and traffic through the LLMInspect API Gateway.

Accessing InspectBoard

Navigate to InspectBoard at:
```
https://your-domain:5119
```
Replace your-domain with the domain you entered during installation. If you use LLMInspect SaaS, open the Observability Portal instead.
Log in using your LLMInspect credentials.

After you log in, InspectBoard opens the area of the dashboard that matches your account's access:
- Administrators see Chat Analytics and API Analytics in the left navigation.
- All other users see My Dashboard, which shows their own activity.

To end your session, click the sign-out button in the top-right corner.

Dashboard Areas

Area	Available to	Purpose
My Dashboard	Users	Your own InspectChat activity and validation outcomes
Chat Analytics	Administrators	InspectChat usage across all users, with the option to focus on a single user
API Analytics	Administrators	Traffic, performance and validation outcomes for the LLMInspect API Gateway and its API clients

Chat Analytics and API Analytics are organised into three collapsible sections:

Summary - Key figures for the selected time range
Analysis - Charts that break the figures down
Detail - Individual messages, users, files and audit logs

My Dashboard

My Dashboard gives each user a personal view of their own activity:

Messages in range - Number of messages you sent in the selected time range
Failed validations - Messages that did not pass one or more GuardRail checks
Image generations (all time) - Total number of images you have generated
Avg tokens / message - Average token usage per message
Foundation model usage - Which AI models you have used
Failed validations by guardrail - Which GuardRails your failed messages were flagged by
Most used model - The model you use most often
Data volume (leakage) - Daily data volume from your activity
Messages & validation outcomes - Your individual messages, with the result of each GuardRail validation
My files - Files you have uploaded

Chat Analytics

Chat Analytics gives administrators a view of InspectChat usage across the organisation. Use the User selector in the Summary section to focus the dashboard on a specific user. The selector includes a search field to quickly find a user by name.

Summary

Total users
Messages in range
Failed validations
Image generations (all time)
Avg tokens / message

Analysis

Foundation model usage - How often each AI model is used
Failed validations by guardrail - Which GuardRails are blocking or flagging messages
Most used model
Data volume (leakage) - Daily data volume, useful for identifying potential data leakage

Detail

Messages & validation outcomes - Each message with its time, content, GuardRail validation results, IP address and user agent. Click a long message to expand it.
InspectChat users - Searchable list of InspectChat users
Files - Files uploaded by the selected user
API Gateway audit logs - See Viewing Audit Logs

API Analytics

API Analytics gives administrators visibility into requests that pass through the LLMInspect API Gateway, including traffic from API clients integrated in Tap Mode. Use the API client selector in the Summary section to focus on a specific API client. API clients are identified using the rules described in User Identification (Tap Mode).

Summary

The Summary section is split into two groups:

Global - Figures for the whole API Gateway, such as total requests, accepted and handled connections, gateway health indicators and total users
Selected client - Messages in range and Failed validations for the selected API client

Analysis

Traffic - Connection states, total bandwidth, and ingress and egress bandwidth by route
Requests - Requests per second by status code and by route
Latency - Request, gateway and upstream latency
Failed validations by guardrail
Data volume (leakage)

Detail

Messages & validation outcomes - Individual messages from the selected API client, with their GuardRail validation results
API Gateway audit logs - See Viewing Audit Logs

Use API Analytics to:

Track traffic volume and request rates over time
Identify performance bottlenecks through latency and status-code trends
Review GuardRail violations for individual API clients
Spot unusual behaviour or spikes in activity

Selecting a Time Range

All dashboards show data for the time range selected in the top bar. The following preset ranges are available:

Last 15 min
Last 1 hour
Last 6 hours
Last 24 hours
Last 7 days
Last 30 days

Time ranges are rolling: the selected range always ends at the current time and moves forward as the dashboard refreshes. Open the time range selector to see the exact start and end times of the current range.

Note: Figures labelled (all time), such as image generations, are not limited to the selected time range.

Viewing Audit Logs

The API Gateway audit logs panel is available in the Detail section of Chat Analytics and API Analytics. It shows audit log lines from the selected time range.

To search the logs:

Type the text you are looking for in the filter lines field of the panel.
Press Enter to show only the log lines that contain that text.
Click a log line to expand it and see the full entry.

To clear the filter, delete the text and press Enter again.

Refreshing Data

Automatic refresh - Dashboard data refreshes automatically every 30 seconds.
Manual refresh - Click the refresh button in the top bar to update all panels immediately.

The status next to the refresh button, for example Updated just now, shows how recently the dashboard received data.