LLMInspect Documentation
Welcome to the official LLMInspect documentation. LLMInspect is an enterprise-grade platform designed to monitor, analyze, and secure your Large Language Model (LLM) API traffic in real-time. Whether you are deploying AI models on-premise or using cloud-based LLM services, LLMInspect provides full visibility, control, and security over your AI infrastructure.
What is LLMInspect?
LLMInspect is an enterprise AI gateway and monitoring solution that sits between your applications and LLM providers such as OpenAI, Azure OpenAI, and other AI services. It provides real-time logging, security guardrails, cost optimization, and enterprise authentication for all your LLM API calls.
Learn about the different components and functionalities of LLMInspect. Learn more about LLMInspect.
Enterprise Architecture
LLMInspect is built on a robust microservice architecture designed for enterprise scalability and reliability. It integrates seamlessly with your existing infrastructure and supports high-availability deployments.
Understand the architectural components and design of LLMInspect. Explore the Enterprise Architecture.
Key Features
LLMInspect offers a comprehensive set of enterprise features including:
- Unified Gateway — Single entry point for all LLM API traffic
- Model Selection — Dynamic routing between different AI models
- Real-time Logging — Complete audit trail of all LLM interactions
- Enterprise Authentication — SSO, LDAP, and role-based access control
- Multimodal Support — Handle text, image, and other modalities
- Desktop App — Native desktop client for easy management
- Security Guardrails (iGuard) — Prevent prompt injection and data leakage
- Cost Optimization — Track and reduce LLM API spending
- Grafana Monitoring — Real-time dashboards and alerts
- Dynamic Config — Hot-reload configuration without downtime
- Splunk Integration — Enterprise log management and SIEM integration
Licensing Guide
Find out about LLMInspect's licensing framework and terms. Read the Licensing Guide.
Troubleshooting Guide
Learn how to resolve common issues and troubleshoot LLMInspect effectively. Our troubleshooting guide covers installation issues, connectivity problems, authentication errors, and performance optimization. Read the Troubleshooting Guide.
Integration Guides
LLMInspect integrates with a wide range of third-party services and databases to extend its capabilities. Supported integrations include Kong API Gateway, MongoDB, Splunk, and more. Integration Guide.
Llama Guard Deployment Guide
Deploy and manage Llama Guard — the critical component ensuring secure guardrails for LLMInspect. Llama Guard provides content moderation and safety filtering for all LLM interactions. Deploy Llama Guard.
LLMInspect SaaS Documentation
Don't want to install LLMInspect on-premise? Access our cloud-hosted SaaS solution and start using secure AI interactions immediately with zero setup required. SaaS Documentation.
LLMInspect On-Premise Documentation
Learn about LLMInspect's on-premise deployment and configuration for organizations that require full data sovereignty and control. Read the On-Premise Documentation.