Installation

LLMInspect is installed with the llminspect-cli installer. The installer asks for a few values, creates the required directories and certificates, configures HTTPS for all services, and starts LLMInspect. You do not need to edit any configuration files.

Prerequisites

A Linux server (the installer runs on Linux only)
Docker Engine (20.10.x or newer)
Docker Compose v2 (the docker compose command)
Minimum 32GB RAM
8+ CPU cores
100GB available storage (SSD recommended)
Internet access to Docker Hub, to download the LLMInspect images
Docker Hub credentials with access to the LLMInspect images
A domain name (FQDN) for the server, such as llminspect.example.com, that your users can resolve
SSL/TLS certificate and private key for that domain (optional; the installer can generate a self-signed certificate)
Network access to required services
A locally running LLM with at-least 70B parameters

Before you start, have the following ready: your MongoDB credentials of choice, a Keycloak administrator password, your OpenAI and Gemini API keys, the Llama Guard and local LLM endpoints, and your LLM Gateway license key and auth code. See Installation Configuration for what each value means.

Installation Steps

Extract the provided zip file and change into the extracted directory:
```
unzip llminspect-onpremise.zip
cd llminspect
```
Run the installer. Choose one of the following options.

Option A: Install with a self-signed certificate (for testing)
```
./llminspect-cli -p /full/path/to/llminspect -o install
```
The installer asks for your domain name and generates a self-signed certificate for it.

Option B: Install with your own certificate (recommended for production)
```
./llminspect-cli -p /full/path/to/llminspect -o install \
    --cert-file /path/to/certificate.crt \
    --cert-key /path/to/private.key \
    --domain llminspect.example.com
```
Use a certificate issued for the domain you pass with --domain. --cert-file, --cert-key and --domain must be used together.

The -p flag must be the full path to the directory you extracted in step 1.
Enter your Docker Hub credentials. The installer first checks that Docker and Docker Compose are available, then asks for the Docker Hub username and password used to download the LLMInspect images.

Note: The password is displayed on screen as you type it.

Enter the configuration values. Under Security Configuration Setup, the installer asks for the following values, one at a time:

Prompt	What to enter
`MONGO_INITDB_ROOT_USERNAME`	A username for the MongoDB administrator, e.g. `mongoadmin`
`MONGO_INITDB_ROOT_PASSWORD`	A strong password for that MongoDB user (hidden while typing)
`KEYCLOAK_ADMIN_PASSWORD`	A strong password for the Keycloak administrator account `admin` (hidden while typing)
`OPENAI_API_KEY`	Your OpenAI API key (hidden while typing)
`GEMINI_API_KEY`	Your Google Gemini API key (hidden while typing)
`LLAMA_GUARD_URL`	The full endpoint of your Llama Guard instance, e.g. `http://192.168.1.100:8080/generate`
`LLM_BASE_URL`	The base URL of your local LLM, ending in `/v1/`, e.g. `http://192.168.1.60:8000/v1/`
`LLMGW_KEY`	Your LLM Gateway license key
`AUTHCODE`	Your LLM Gateway auth code

The installer then generates the remaining internal secrets automatically. See Installation Configuration for details and examples.

Enter your domain name (Option A only). When prompted with Please enter your Fully Qualified Domain Name (FQDN), enter the domain name of the server, e.g. llminspect.example.com. Enter the name only, without https:// or a port. All LLMInspect services are published on this domain.
Review the configuration summary. The installer prints the domain and certificate paths, followed by the full configuration.

Note: The configuration shown on screen includes the passwords and API keys you entered. Do not share this output.
Wait for the installation to finish. The installer saves the configuration to a .env file in the installation directory, starts all services with Docker Compose, and prints Installation completed successfully! when it is done.

After the installation, open LLMInspect using the URLs in Accessing LLMInspect.

What the Installer Configures Automatically

You do not need to configure any of the following manually:

The directories LLMInspect needs for logs and images, with the correct permissions
The Docker logging driver used for LLMInspect logs, if it is not already installed
A self-signed certificate for your domain (Option A), saved in the certificates directory of the installation
HTTPS for all services, using your certificate or the generated one
Your domain in the service and sign-in (Keycloak) configuration
Internal secrets, such as the API gateway keys, authentication client secrets and the Keycloak database credentials

Certificates

Self-signed certificate (Option A): The installer generates a certificate for the domain you enter. It is valid for 365 days. Browsers show a security warning for self-signed certificates, so use this option for testing only.
Your own certificate (Option B): Provide the certificate and private key files with --cert-file and --cert-key, and the domain the certificate was issued for with --domain. For production, obtain certificates from a trusted Certificate Authority (CA).

⚠️ Self-signed certificates (generated in basic installation) are suitable for testing but not recommended for production use. For production, please obtain certificates from a trusted Certificate Authority (CA).

Using an Environment File

To avoid entering the configuration values interactively, for example in automated deployments, provide them in an environment file with --env-file:

./llminspect-cli -p /full/path/to/llminspect -o install \
    --cert-file /path/to/certificate.crt \
    --cert-key /path/to/private.key \
    --domain llminspect.example.com \
    --env-file /path/to/llminspect.env

The file uses KEY=VALUE lines and must contain all nine values listed in step 4. See Installation Configuration for an example and for the additional values you should include. The installer still asks for your Docker Hub credentials, and for your domain name if you do not use --cert-file, --cert-key and --domain.

Important Notes

The installer should only be used for the initial installation. If the installation directory already contains a .env file, the installer stops without making changes.
After installation, use standard Docker Compose commands in the installation directory for daily operations, such as docker compose up -d to start and docker compose down to stop services. See Daily Operations.
The installer writes a detailed log to install.log in the directory you run it from. This log contains the configuration values you entered, so protect or delete it after installation.

⚠️ IMPORTANT: If Installation Fails ⚠️

If the installer encounters an error and fails to complete, you MUST run the clean operation before attempting to install again:

./llminspect-cli -p /full/path/to/llminspect -o clean

The clean operation asks for confirmation, removes all LLMInspect containers and their volumes, and restores the configuration files in the installation directory to their original state. It also offers to prune unused Docker resources.

Parameter Summary:

-p, --project: (Required) Full path to the extracted llminspect directory
-o, --operation: (Required) One of:
install: Performs initial installation
clean: Removes all containers and volumes and restores the installation directory to its original state. Useful for:
- Starting fresh after a failed installation
- Recovering from installer-generated configuration errors
- Resetting the environment before attempting a new installation
- Cleaning up partial changes if installation was interrupted
--env-file: (Optional) Path to environment file for automated configuration
SSL/TLS group (all three required if using your own certificate):
--domain: Domain name the certificate was issued for
--cert-file: Path to SSL certificate
--cert-key: Path to SSL private key
--license: (Optional) To see the license of open-source services used in llminspect