Skip to content

🚀InspectRAG Feature Documentation

Table of Contents

  1. Introduction
  2. Overview of the InspectRAG Feature
  3. How InspectRAG Works
    • Active Directory Integration
    • Real-Time File Uploads and Synchronization
    • Role-Based Access Control
    • Permission Management
    • Authentication with Keycloak
  4. Key Benefits and Significance in the Industry
    • Enhanced Security 🔒
    • Real-Time Data Accessibility ⏱️
    • Scalability and Flexibility 🔄
    • Compliance and Governance 📋
  5. Implementation Details
    • Supported Environments
    • Integration Steps
    • Technical Architecture
  6. Security Features and Mechanisms
    • Data Encryption
    • Multi-Factor Authentication (MFA)
    • Audit Logging and Monitoring
    • Threat Detection and Response
  7. Visual Representations
    • System Architecture Diagram
    • Access Control Flowchart
    • Tables and Data Structures
  8. Use Cases
    • Enterprise Document Management
    • Secure Collaboration
    • Regulatory Compliance
  9. Conclusion
  10. References

Introduction

In today's digital era, organizations handle an enormous volume of sensitive data that requires stringent security measures and efficient access controls. Ensuring that employees access only the information they are authorized to view is crucial for maintaining data integrity and compliance with industry regulations. Introducing InspectRAG 🎯—a groundbreaking feature of LLM Inspect by Eunomatix.us. InspectRAG is a Role-Based Access Control (RBAC) system that seamlessly integrates with Active Directory and provides real-time file uploads for querying, all while emphasizing top-tier security.

Overview of the InspectRAG Feature

InspectRAG stands for Inspect Retrieval-Augmented Generation, a feature designed to:

  • 🔗 Link with Active Directory: Sync user and group permissions directly.
  • 📁 Enable Real-Time File Uploads: Keep your data up-to-date for immediate querying.
  • 🛡️ Implement Role-Based Access Control: Restrict access based on user roles and permissions.
  • 🌐 Support Both On-Premise and Online SharePoint: Flexible deployment options to suit organizational needs.
  • 🔑 Authenticate via Keycloak: Secure user authentication and role management.

How InspectRAG Works

Active Directory Integration

InspectRAG connects with your organization's Active Directory to:

  • Authenticate Users: Validate user credentials against the directory.
  • Fetch User Roles and Groups: Import roles and group memberships for access control.
  • Sync Permissions: Ensure that any changes in Active Directory are reflected in real-time.

Benefits:

  • Centralized User Management: Simplifies administration by managing users in one place.
  • Consistency: Maintains uniformity in user roles and permissions across systems.

Real-Time File Uploads and Synchronization

The system ensures that:

  • Files are Extracted in Real-Time: Keeps the vector database synchronized with the latest documents.
  • Data Consistency is Maintained: Any changes in SharePoint are immediately updated.
  • Permissions are Updated: Reflects any modifications in file permissions instantaneously.

Benefits:

  • Up-to-Date Data: Users always access the most recent information.
  • Reduced Latency: Eliminates delays in data availability.

Role-Based Access Control

Access to files and querying capabilities are governed by:

  • User Roles: Permissions assigned based on the user's position and responsibilities.
  • Group Permissions: Collective access rights for users within the same group.
  • Hierarchical Access Levels: Allows for nested permissions and role inheritance.

Benefits:

  • Granular Control: Fine-tuned access permissions enhance security.
  • Scalability: Easily manage permissions for large organizations.

Permission Management

Permissions are:

  • Stored in the Vector Database: Alongside file data for quick access during queries.
  • Updated in Real-Time: Any changes in SharePoint permissions are immediately reflected.
  • Cross-Verified: During query time, user roles in Keycloak are matched with permissions in the database and SharePoint.

Benefits:

  • Enhanced Security: Prevents unauthorized access.
  • Compliance: Adheres to data protection regulations.

Authentication with Keycloak

Keycloak provides:

  • Secure Authentication: Users log in through a trusted system.
  • Role Management: Assigns and manages user roles and permissions.
  • Integration with Active Directory: Syncs user data for consistent access control.

Benefits:

  • Single Sign-On (SSO): Streamlines the login process.
  • Security: Robust authentication mechanisms protect against breaches.

Key Benefits and Significance in the Industry

Enhanced Security 🔒

  • Multi-Layered Access Control: Combines Active Directory roles with Keycloak authentication.
  • Real-Time Permission Updates: Immediate reflection of any changes in user permissions.
  • Data Encryption: Protects data both at rest and in transit.

Why It Matters:

In an age where data breaches are common, having a system that proactively secures sensitive information is invaluable. InspectRAG's robust security measures help organizations protect their assets and maintain trust with clients and stakeholders.

Real-Time Data Accessibility ⏱️

  • Instant Data Availability: Users access the most current data without delay.
  • Efficient Workflows: Enhances productivity by reducing waiting times for data synchronization.
  • Responsive Querying: Immediate results from data queries improve decision-making processes.

Why It Matters:

In fast-paced business environments, timely access to information is critical. InspectRAG ensures that teams have the data they need when they need it, fostering agility and responsiveness.

Scalability and Flexibility 🔄

  • Supports Various Environments: Whether on-premise or cloud-based, InspectRAG adapts to your infrastructure.
  • Flexible Integration: Works with existing systems and can be customized to fit organizational needs.
  • Future-Proof Design: Built to accommodate growth and technological advancements.

Why It Matters:

Organizations evolve, and so do their technological needs. A scalable and flexible system like InspectRAG ensures long-term viability and return on investment.

Compliance and Governance 📋

  • Audit Trails: Comprehensive logs for tracking access and modifications.
  • Policy Enforcement: Ensures organizational policies are adhered to consistently.

Why It Matters:

Non-compliance can result in hefty fines and legal repercussions .InspectRAG aids in maintaining compliance, thereby mitigating risks associated with regulatory breaches.

Using InspectRAG: Step-by-Step Guide

Step 1: Log in with OpenID Credentials

Users begin by logging into the system using their OpenID credentials. This ensures secure authentication through a trusted identity provider.

Example:

  • Username: janedoe@eunomatix.com
  • Password: **

Once authenticated, the user is redirected to the InspectChat page for further actions.

Step 2: Authenticate to the InspectChat Page

After successful login, users are redirected to the InspectChat page, the central interface where they can access various plugins, including InspectRAG.

Example:

  • janedoe logs in and lands on the InspectChat interface.
  • He sees the InspectRAG plugin in the list of available tools.

Step 3: Select the InspectRAG Plugin

Users select the InspectRAG plugin from the list to begin querying documents securely with role-based access.

Example:

  • janedoe clicks on the InspectRAG plugin from the plugin menu.
  • The plugin loads, ready to accept input for document queries.

Step 4: Enter a Query Prompt

Users provide a prompt describing the document or data they want to access. The prompt helps the system find relevant files stored in SharePoint.

Example:

  • janedoe enters the following query:

    Show me the latest project report for the marketing team.

The system begins processing the query to identify the relevant documents.

Step 5: Match User Roles and Permissions with File Metadata

Before granting access to the document, InspectRAG cross-verifies the user’s roles, groups, or username from Active Directory with the permissions, creator, and access roles of the document stored in SharePoint.

Example 1: Access Granted (Matching Roles)

  1. User Metadata from Active Directory:

    {
  "user_id": "janedoe",
  "username": "janedoe",
  "roles": ["WSS_WPG", "Marketing Team"],
  "groups": ["Home Owners"]
}

  2. Document Metadata in SharePoint:

    {
  "file_id": "marketing_report_2024",
  "permissions": ["Marketing Team", "WSS_WPG"],
  "creator": "jane.doe",
  "last_modified": "2024-10-10"
}

In this example:

  • janedoe’s roles and groups match the document’s permissions.
  • Since there is a match, the system grants access to the document.

Response:

Here is the latest marketing report: [Marketing Report 2024]

Example 2: Access Denied (Mismatched Roles)

  1. User Metadata from Active Directory:

    {
  "user_id": "janedoe",
  "username": "janedoe",
  "roles": ["WSS_WPG", "Home Members"],
  "groups": ["Home Owners"]
}

  2. Document Metadata in SharePoint:

    {
  "file_id": "finance_report_2024",
  "permissions": ["Finance Team"],
  "creator": "finance.admin",
  "last_modified": "2024-09-30"
}

In this example:

  • janedoe’s roles and groups do not match the required Finance Team permissions for the file.
  • As a result, the system blocks access to the document.

Blocked Response:

You don't have permissions to access the contents of the said file, please contact your administrator to get rights.

End-to-End Example Flow

  1. Login: janedoe logs in using his OpenID credentials.
  2. Authentication: He is redirected to the InspectChat page.
  3. Plugin Selection: janedoe selects the InspectRAG plugin.
  4. Enter Query: He enters a query for a specific marketing report.
  5. Role Matching: InspectRAG checks his Active Directory roles, groups, and username against the permissions and creator of the document stored in SharePoint.
  6. Access Decision:
    • If the roles match, the document is returned.
    • If the roles don’t match, access is blocked with a message.

Summary of Role and Permission Matching

  • Matching Criteria:
    • The user’s roles or groups match the permissions on the document.
    • OR the user’s username matches the creator or permissions of the document.

  • Blocked Access Message:

    If no match is found the following response is given:

    image.png

This ensures that users only access the documents they are authorized to view, maintaining strict role-based access control (RBAC) for secure document management.

Implementation Details

Supported Environments

Component Supported Platforms
SharePoint On-Premise, Online
Active Directory Windows Server AD, Azure AD
Authentication Keycloak
Databases Vector Databases (PgVectorDB)

Integration Steps

  1. Active Directory Setup: Configure InspectRAG to connect with your Active Directory using LDAP or appropriate protocols.
  2. Keycloak Configuration: Set up Keycloak realms, clients, and users to manage authentication.
  3. SharePoint Connection: Establish secure connections to your SharePoint sites using appropriate APIs.
  4. Vector Database Initialization: Set up and configure your vector database for storing file data and permissions.
  5. Real-Time Synchronization: Implement webhooks or polling mechanisms to monitor file and permission changes.
  6. Testing and Validation: Conduct thorough testing to ensure all components function as expected.

Architecture Diagram

!https://prod-files-secure.s3.us-west-2.amazonaws.com/a8646d55-ea1e-440f-9271-be22b85e3ab5/652b6c84-f1d2-4659-86c8-f65ea251ba13/InspectRAG_diagram_with_bg.png

Security Features and Mechanisms

Audit Logging and Monitoring

  • Comprehensive Logs: Records all access attempts, successful or otherwise.
  • Real-Time Monitoring: Alerts administrators of suspicious activities.

Threat Detection and Response

  • Anomaly Detection: Uses machine learning algorithms to detect unusual patterns.

Metadata Stored with File Embeddings

There are different metadata stored with each chunk of the file that allows to ensure that the entire file has permission locks on it , increasing security of the RBAC. The metadata of each file includes the following:

  • file_id: A unique identifier assigned to the file within the system.
  • user_id: Identifies the user or access level associated with the file.
  • digest: A numerical hash to ensure content or file size detect file content changes.
  • source: The original file path indicating where the document is stored or accessed.
  • page: Specifies the page number currently being processed or referenced.
  • role: The role or access level associated with the document.
  • permissions: Lists the users or groups allowed to access the document.
  • creator: Identifies the creator of the file or metadata entry.
  • last_modified: Records the timestamp of the last modification of the file.

Use Cases

Enterprise Document Management

Scenario: A multinational corporation needs to manage documents across various departments.

  • Challenge: Ensuring only authorized personnel access sensitive documents.
  • Solution: InspectRAG provides role-based access, real-time synchronization, and audit logs.

Secure Collaboration

Scenario: A team working on a confidential project requires a secure platform for collaboration.

  • Challenge: Preventing data leaks while allowing seamless teamwork.
  • Solution: InspectRAG's secure authentication and access controls enable safe collaboration.

Frequently Asked Questions (FAQs)

Q1: How does InspectRAG integrate with Active Directory?

A: InspectRAG uses LDAP to fetch users and groups from Active Directory. It synchronizes permissions and roles in real-time to ensure access control is consistent across the organization.

Q2: Can InspectRAG be deployed in both on-premise and cloud environments?

A: Yes, InspectRAG supports hybrid environments, including on-premise deployments with Windows Active Directory and cloud-based Azure AD and SharePoint Online.

Q3: What happens if user roles or permissions change in Active Directory?

A: InspectRAG synchronizes permissions in real-time. If any changes occur in Active Directory, they are immediately reflected in InspectRAG during the next sync cycle to ensure access remains up-to-date.

Q4: How does InspectRAG ensure secure access to documents?

A: InspectRAG enforces Role-Based Access Control (RBAC) by cross-verifying user roles in Active Directory and permissions stored in SharePoint. Users can only access documents they are authorized to query.

Q5: What happens if the file content or permissions are modified after embedding?

A: Any changes in file permissions or content are captured through real-time synchronization with the vector database and SharePoint, ensuring users access the most current and authorized version of the document.

Q6: What types of databases are supported for storing embeddings?

A: InspectRAG currently supports PgVectorDB as the vector database for storing document embeddings and metadata. The system can be extended to support additional databases if required.

Q7: How are audit logs generated and stored?

A: All access attempts, modifications, and queries are logged and stored in audit logs. These logs can be monitored in real-time to detect suspicious activities and are essential for compliance reporting.

Q8: Can InspectRAG be customized for specific organizational needs?

A: Yes, the system is flexible and can be customized to meet the specific requirements of your organization. InspectRAG can integrate with existing systems and adapt to evolving needs over time.

Q10: How is matching done between the file and user permissions?

A: When a user sends a query, InspectRAG matches the user's roles, groups, or username with the permissions, creator, and access roles defined for the file in SharePoint. If a match is found, the user is granted access. If not, the following message is shown:

You don't have permissions to access the contents of the said file, please contact your administrator to get rights.

Conclusion

InspectRAG by Eunomatix.us is more than just a feature; it's a comprehensive solution that addresses the critical needs of modern organizations for secure, efficient, and compliant data management. By integrating robust security measures, real-time data synchronization, and flexible access controls, InspectRAG sets a new standard in the industry. Its emphasis on security 🔒, real-time accessibility ⏱️, scalability 🔄, and compliance 📋 makes it an indispensable tool for organizations aiming to protect their data assets and streamline operations.

References

For implementation assistance or further inquiries about the InspectRAG feature, please contact the Eunomatix support team at support@eunomatix.us.