feat: Implement multi-tenant support across graph and query routes
- Enhanced graph_routes.py and query_routes.py to support multi-tenant architecture by introducing tenant-specific RAG instances. - Updated create_graph_routes and create_query_routes functions to accept rag_manager for tenant management. - Added get_tenant_rag dependency to all relevant endpoints to ensure tenant context is utilized for operations. - Modified Vite configuration to include comprehensive API proxy rules for seamless interaction with backend services. - Implemented cascade delete functionality in tenant_service.py for tenant and knowledge base deletions. - Added detailed logging and error handling for tenant operations. - Created audit logs documenting the multi-tenant implementation process and decisions made.
This commit is contained in:
Raphaël MANSUY
parent
a6aa073d70
commit
730c406749
14 changed files with 1109 additions and 89 deletions
8
.vite/deps/_metadata.json
Normal file
8
.vite/deps/_metadata.json
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
{
|
||||
"hash": "1c2bee50",
|
||||
"configHash": "21727160",
|
||||
"lockfileHash": "e3b0c442",
|
||||
"browserHash": "8cd912e5",
|
||||
"optimized": {},
|
||||
"chunks": {}
|
||||
}
|
||||
3
.vite/deps/package.json
Normal file
3
.vite/deps/package.json
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
{
|
||||
"type": "module"
|
||||
}
|
||||
381
docs/0009-multi-tenant-vs-workspace-audit.md
Normal file
381
docs/0009-multi-tenant-vs-workspace-audit.md
Normal file
|
|
@ -0,0 +1,381 @@
|
|||
# Multi-Tenant vs Workspace Architecture Audit Report
|
||||
|
||||
**Date:** 2024-12-05
|
||||
**Status:** ✅ PASSED - No Redundancy Found
|
||||
**Author:** AI Audit Agent
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This audit evaluates whether the **Multi-Tenant feature** (local HKU implementation) is redundant with the **Workspace feature** (upstream HKUDS/LightRAG).
|
||||
|
||||
**Verdict: NOT REDUNDANT** - The features serve different purposes in a well-designed layered architecture:
|
||||
|
||||
| Feature | Layer | Purpose |
|
||||
|---------|-------|---------|
|
||||
| **Workspace** (upstream) | Storage Layer | Low-level data isolation mechanism in database tables |
|
||||
| **Tenant** (local) | Application Layer | High-level multi-tenant SaaS with user management, RBAC, and APIs |
|
||||
|
||||
The Tenant feature **extends and uses** the Workspace feature - it's a proper abstraction layer, not duplication.
|
||||
|
||||
---
|
||||
|
||||
## 1. Workspace Feature (Upstream LightRAG)
|
||||
|
||||
### 1.1 Purpose
|
||||
The `workspace` parameter in LightRAG provides **storage-level data isolation** between different LightRAG instances.
|
||||
|
||||
### 1.2 Implementation
|
||||
|
||||
**Core Parameter:**
|
||||
```python
|
||||
# From lightrag/lightrag.py
|
||||
@dataclass
|
||||
class LightRAG:
|
||||
workspace: str = field(default_factory=lambda: os.getenv("WORKSPACE", ""))
|
||||
"""Workspace for data isolation. Defaults to empty string if WORKSPACE environment variable is not set."""
|
||||
```
|
||||
|
||||
**Storage Isolation:**
|
||||
All storage classes receive the `workspace` parameter and use it in their primary keys:
|
||||
|
||||
```python
|
||||
# From lightrag/lightrag.py - storage initialization
|
||||
self.llm_response_cache = self.key_string_value_json_storage_cls(
|
||||
namespace=NameSpace.KV_STORE_LLM_RESPONSE_CACHE,
|
||||
workspace=self.workspace, # Passed to all storages
|
||||
...
|
||||
)
|
||||
```
|
||||
|
||||
**Database Schema (PostgreSQL):**
|
||||
```sql
|
||||
-- Every LIGHTRAG_* table has workspace in PRIMARY KEY
|
||||
CREATE TABLE LIGHTRAG_DOC_FULL (
|
||||
id VARCHAR(255),
|
||||
workspace VARCHAR(255),
|
||||
...
|
||||
CONSTRAINT LIGHTRAG_DOC_FULL_PK PRIMARY KEY (workspace, id)
|
||||
);
|
||||
```
|
||||
|
||||
### 1.3 Environment Variables
|
||||
|
||||
| Variable | Storage Type | Description |
|
||||
|----------|-------------|-------------|
|
||||
| `WORKSPACE` | Generic | Default workspace for all storages |
|
||||
| `POSTGRES_WORKSPACE` | PostgreSQL | PostgreSQL-specific workspace |
|
||||
| `REDIS_WORKSPACE` | Redis | Redis-specific workspace |
|
||||
| `MONGODB_WORKSPACE` | MongoDB | MongoDB-specific workspace |
|
||||
| `MILVUS_WORKSPACE` | Milvus | Milvus-specific workspace |
|
||||
| `QDRANT_WORKSPACE` | Qdrant | Qdrant-specific workspace |
|
||||
| `NEO4J_WORKSPACE` | Neo4j | Neo4j-specific workspace |
|
||||
|
||||
### 1.4 Limitations
|
||||
|
||||
The workspace feature provides **only storage isolation**:
|
||||
- ❌ No user management
|
||||
- ❌ No authentication/authorization
|
||||
- ❌ No CRUD API for workspace management
|
||||
- ❌ No metadata or descriptions
|
||||
- ❌ No UI support
|
||||
- ❌ No concept of multiple knowledge bases per workspace
|
||||
|
||||
---
|
||||
|
||||
## 2. Multi-Tenant Feature (Local Implementation)
|
||||
|
||||
### 2.1 Purpose
|
||||
The Multi-Tenant feature provides a **complete SaaS multi-tenancy layer** on top of LightRAG, including:
|
||||
- Organization (tenant) management
|
||||
- Multiple knowledge bases per tenant
|
||||
- Role-based access control (RBAC)
|
||||
- User-tenant membership
|
||||
- REST API for management
|
||||
- WebUI for tenant/KB selection
|
||||
|
||||
### 2.2 Key Components
|
||||
|
||||
| Component | File | Purpose |
|
||||
|-----------|------|---------|
|
||||
| **Tenant Model** | `lightrag/models/tenant.py` | Data models for Tenant, KnowledgeBase, TenantContext |
|
||||
| **TenantService** | `lightrag/services/tenant_service.py` | CRUD operations, access verification |
|
||||
| **TenantRAGManager** | `lightrag/tenant_rag_manager.py` | Manages RAG instances per tenant/KB |
|
||||
| **Tenant Routes** | `lightrag/api/routers/tenant_routes.py` | REST API endpoints |
|
||||
| **Security** | `lightrag/security.py` | Validation, path traversal prevention |
|
||||
|
||||
### 2.3 How Tenant Uses Workspace
|
||||
|
||||
**Critical Integration Point:**
|
||||
|
||||
```python
|
||||
# From lightrag/tenant_rag_manager.py
|
||||
async def get_rag_instance(self, tenant_id: str, kb_id: str, user_id: str):
|
||||
# SECURITY: Validate identifiers
|
||||
tenant_id = validate_identifier(tenant_id, "tenant_id")
|
||||
kb_id = validate_identifier(kb_id, "kb_id")
|
||||
|
||||
# Create composite workspace
|
||||
tenant_working_dir, composite_workspace = validate_working_directory(
|
||||
self.base_working_dir, tenant_id, kb_id
|
||||
)
|
||||
# composite_workspace = f"{tenant_id}:{kb_id}"
|
||||
|
||||
# Create RAG instance with composite workspace
|
||||
instance = LightRAG(
|
||||
working_dir=tenant_working_dir,
|
||||
workspace=composite_workspace, # Uses workspace under the hood!
|
||||
...
|
||||
)
|
||||
```
|
||||
|
||||
**The Tenant feature DELEGATES to Workspace for actual data isolation.**
|
||||
|
||||
### 2.4 Database Schema
|
||||
|
||||
**Management Tables (Tenant Layer):**
|
||||
```sql
|
||||
-- Tenant metadata
|
||||
CREATE TABLE tenants (
|
||||
tenant_id VARCHAR(255) UNIQUE NOT NULL,
|
||||
name VARCHAR(255) NOT NULL,
|
||||
description TEXT,
|
||||
metadata JSONB,
|
||||
...
|
||||
);
|
||||
|
||||
-- Knowledge bases within tenants
|
||||
CREATE TABLE knowledge_bases (
|
||||
tenant_id VARCHAR(255) REFERENCES tenants(tenant_id),
|
||||
kb_id VARCHAR(255) NOT NULL,
|
||||
name VARCHAR(255) NOT NULL,
|
||||
...
|
||||
);
|
||||
|
||||
-- User access control
|
||||
CREATE TABLE user_tenant_memberships (
|
||||
user_id VARCHAR(255) NOT NULL,
|
||||
tenant_id VARCHAR(255) REFERENCES tenants(tenant_id),
|
||||
role VARCHAR(50) NOT NULL, -- owner, admin, editor, viewer
|
||||
...
|
||||
);
|
||||
```
|
||||
|
||||
**Generated Columns for Integration:**
|
||||
```sql
|
||||
-- LIGHTRAG_* tables have generated columns to extract tenant/kb
|
||||
ALTER TABLE LIGHTRAG_DOC_FULL ADD COLUMN
|
||||
tenant_id VARCHAR(255) GENERATED ALWAYS AS (
|
||||
CASE WHEN workspace LIKE '%:%'
|
||||
THEN SPLIT_PART(workspace, ':', 1)
|
||||
ELSE workspace END
|
||||
) STORED,
|
||||
kb_id VARCHAR(255) GENERATED ALWAYS AS (
|
||||
CASE WHEN workspace LIKE '%:%'
|
||||
THEN SPLIT_PART(workspace, ':', 2)
|
||||
ELSE 'default' END
|
||||
) STORED;
|
||||
```
|
||||
|
||||
This allows querying data by tenant/KB without modifying the core storage implementation.
|
||||
|
||||
### 2.5 Roles and Permissions
|
||||
|
||||
| Role | Permissions |
|
||||
|------|-------------|
|
||||
| **Owner** | Full control, manage members, delete tenant |
|
||||
| **Admin** | Create/delete KBs, manage documents |
|
||||
| **Editor** | Create/update/delete documents, run queries |
|
||||
| **Viewer** | Read documents, run queries |
|
||||
|
||||
---
|
||||
|
||||
## 3. Architecture Comparison
|
||||
|
||||
### 3.1 Feature Matrix
|
||||
|
||||
| Aspect | Workspace (Upstream) | Tenant (Local) |
|
||||
|--------|---------------------|----------------|
|
||||
| Data Isolation | ✅ Storage-level | ✅ Uses workspace |
|
||||
| User Management | ❌ | ✅ Full RBAC |
|
||||
| Authentication | ❌ | ✅ JWT tokens |
|
||||
| Authorization | ❌ | ✅ Role-based |
|
||||
| CRUD API | ❌ | ✅ REST endpoints |
|
||||
| Multiple KBs | ❌ One per workspace | ✅ Many per tenant |
|
||||
| Configuration | ❌ Global only | ✅ Per-tenant |
|
||||
| Quotas/Limits | ❌ | ✅ Per-tenant |
|
||||
| Metadata | ❌ | ✅ Rich metadata |
|
||||
| UI Support | ❌ | ✅ Selection UI |
|
||||
| File Storage | ✅ Subdirectories | ✅ Uses subdirs |
|
||||
| Backward Compatible | ✅ | ✅ Single-tenant mode |
|
||||
|
||||
### 3.2 Layered Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ WebUI / REST API │
|
||||
│ - Tenant/KB selection │
|
||||
│ - Document upload, query interface │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ Authentication Layer │
|
||||
│ - JWT token validation │
|
||||
│ - User session management │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ Authorization Layer │
|
||||
│ - TenantService.verify_user_access() │
|
||||
│ - Role-based permission checks │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ TenantRAGManager (Instance Cache) │
|
||||
│ - Manages per-tenant/KB LightRAG instances │
|
||||
│ - LRU eviction for memory management │
|
||||
│ - Creates composite_workspace = "{tenant}:{kb}" │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ LightRAG Core │
|
||||
│ - Uses workspace for storage isolation │
|
||||
│ - KV, Vector, Graph, DocStatus storages │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ PostgreSQL / Storage Backend │
|
||||
│ - PRIMARY KEY (workspace, id) for isolation │
|
||||
│ - Generated columns extract tenant_id, kb_id │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Findings
|
||||
|
||||
### 4.1 No Redundancy Found ✅
|
||||
|
||||
The Tenant feature is **complementary**, not redundant:
|
||||
|
||||
1. **Workspace** = Storage mechanism (HOW data is isolated)
|
||||
2. **Tenant** = Application layer (WHO can access WHAT data)
|
||||
|
||||
They work together:
|
||||
```
|
||||
User Request → Tenant Auth → TenantRAGManager → workspace="{tenant}:{kb}" → Storage
|
||||
```
|
||||
|
||||
### 4.2 Design Quality Assessment
|
||||
|
||||
| Criterion | Score | Notes |
|
||||
|-----------|-------|-------|
|
||||
| Separation of Concerns | ⭐⭐⭐⭐⭐ | Clean layered architecture |
|
||||
| Code Reuse | ⭐⭐⭐⭐⭐ | Tenant uses workspace, doesn't duplicate |
|
||||
| Security | ⭐⭐⭐⭐ | Validation, RBAC, path traversal prevention |
|
||||
| Backward Compatibility | ⭐⭐⭐⭐⭐ | Single-tenant mode still works |
|
||||
| Database Design | ⭐⭐⭐⭐ | Generated columns enable efficient queries |
|
||||
|
||||
### 4.3 Positive Design Decisions
|
||||
|
||||
1. **Composite Workspace Format:** Using `{tenant_id}:{kb_id}` as workspace allows multiple KBs per tenant while reusing storage isolation
|
||||
|
||||
2. **Generated Columns:** PostgreSQL generated columns (`tenant_id`, `kb_id`) enable efficient queries without schema changes to core tables
|
||||
|
||||
3. **Instance Caching:** TenantRAGManager caches RAG instances with LRU eviction for performance
|
||||
|
||||
4. **Security Validation:** `validate_identifier()` and `validate_working_directory()` prevent injection and path traversal
|
||||
|
||||
5. **Environment Toggle:** `LIGHTRAG_MULTI_TENANT` allows switching between single-tenant and multi-tenant modes
|
||||
|
||||
---
|
||||
|
||||
## 5. Recommendations
|
||||
|
||||
### 5.1 Improvements Needed
|
||||
|
||||
| Priority | Issue | Recommendation |
|
||||
|----------|-------|----------------|
|
||||
| **High** | Cascade Delete | Add cleanup of LIGHTRAG_* tables when tenant is deleted |
|
||||
| **Medium** | Documentation | Document workspace naming convention clearly |
|
||||
| **Medium** | Orphan Prevention | Add DB triggers to validate tenant/kb exists on insert |
|
||||
| **Low** | Naming Clarity | Consider renaming `workspace` to `isolation_key` in docs |
|
||||
|
||||
### 5.2 Implementation: Cascade Delete
|
||||
|
||||
Add this to `TenantService.delete_tenant()`:
|
||||
|
||||
```python
|
||||
async def delete_tenant(self, tenant_id: str) -> bool:
|
||||
# Existing: delete KBs
|
||||
kbs_result = await self.list_knowledge_bases(tenant_id)
|
||||
for kb in kbs_result.get("items", []):
|
||||
await self.delete_knowledge_base(tenant_id, kb.kb_id)
|
||||
|
||||
# NEW: Clean up LIGHTRAG_* tables
|
||||
if hasattr(self.kv_storage, 'db') and self.kv_storage.db:
|
||||
await self.kv_storage.db.execute(
|
||||
"DELETE FROM LIGHTRAG_DOC_FULL WHERE workspace LIKE $1",
|
||||
[f"{tenant_id}:%"]
|
||||
)
|
||||
# Repeat for other LIGHTRAG_* tables...
|
||||
|
||||
# Existing: delete tenant metadata
|
||||
await self.kv_storage.delete([f"{self.tenant_namespace}:{tenant_id}"])
|
||||
return True
|
||||
```
|
||||
|
||||
### 5.3 Documentation Update
|
||||
|
||||
Add this to README or multi-tenancy docs:
|
||||
|
||||
```markdown
|
||||
## Workspace vs Multi-Tenant
|
||||
|
||||
LightRAG supports two isolation modes:
|
||||
|
||||
### Single-Tenant Mode (Default)
|
||||
- Set `WORKSPACE=myworkspace` environment variable
|
||||
- All data stored under one workspace
|
||||
- No authentication required
|
||||
|
||||
### Multi-Tenant Mode
|
||||
- Set `LIGHTRAG_MULTI_TENANT=true`
|
||||
- Workspace format: `{tenant_id}:{kb_id}`
|
||||
- Full authentication and RBAC
|
||||
- Multiple knowledge bases per tenant
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Conclusion
|
||||
|
||||
**The Multi-Tenant implementation is well-designed and NOT redundant with the Workspace feature.**
|
||||
|
||||
The architecture correctly layers:
|
||||
1. **Workspace (upstream)** for storage-level isolation
|
||||
2. **Tenant (local)** for application-level multi-tenancy
|
||||
|
||||
This follows best practices for extending open-source projects:
|
||||
- Minimal changes to core code
|
||||
- Clear abstraction layers
|
||||
- Backward compatibility maintained
|
||||
|
||||
**Recommendation:** Approve the current implementation with minor improvements for cascade delete and documentation clarity.
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: File Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `lightrag/lightrag.py` | Core LightRAG class with workspace parameter |
|
||||
| `lightrag/kg/postgres_impl.py` | PostgreSQL storage with workspace in PK |
|
||||
| `lightrag/models/tenant.py` | Tenant, KnowledgeBase, TenantContext models |
|
||||
| `lightrag/services/tenant_service.py` | Tenant/KB CRUD, access verification |
|
||||
| `lightrag/tenant_rag_manager.py` | RAG instance management per tenant/KB |
|
||||
| `lightrag/api/routers/tenant_routes.py` | REST API for tenant management |
|
||||
| `lightrag/security.py` | Identifier validation, security utilities |
|
||||
| `starter/init-postgres.sql` | Database schema with generated columns |
|
||||
|
||||
## Appendix B: Environment Variables
|
||||
|
||||
### Workspace Variables (Upstream)
|
||||
- `WORKSPACE` - Default workspace name
|
||||
- `POSTGRES_WORKSPACE` - PostgreSQL-specific workspace
|
||||
- `REDIS_WORKSPACE` - Redis-specific workspace
|
||||
- `MONGODB_WORKSPACE` - MongoDB-specific workspace
|
||||
|
||||
### Tenant Variables (Local)
|
||||
- `LIGHTRAG_MULTI_TENANT` - Enable multi-tenant mode (true/false)
|
||||
- `LIGHTRAG_SUPER_ADMIN_USERS` - Comma-separated super admin usernames
|
||||
- `REQUIRE_USER_AUTH` - Require authentication (true/false)
|
||||
|
|
@ -52,6 +52,9 @@ from lightrag.api.routers.document_routes import (
|
|||
from lightrag.api.routers.query_routes import create_query_routes
|
||||
from lightrag.api.routers.graph_routes import create_graph_routes
|
||||
from lightrag.api.routers.ollama_api import OllamaAPI
|
||||
from lightrag.api.routers.tenant_routes import create_tenant_routes
|
||||
from lightrag.services.tenant_service import TenantService
|
||||
from lightrag.tenant_rag_manager import TenantRAGManager
|
||||
|
||||
from lightrag.utils import logger, set_verbose_debug
|
||||
from lightrag.kg.shared_storage import (
|
||||
|
|
@ -848,21 +851,55 @@ def create_app(args):
|
|||
logger.error(f"Failed to initialize LightRAG: {e}")
|
||||
raise
|
||||
|
||||
# Add routes
|
||||
# Initialize multi-tenant components if enabled
|
||||
# NOTE: These are initialized here but need the db pool to be ready before use.
|
||||
# The tenant_service uses rag.full_docs.db for database access (initialized in lifespan).
|
||||
tenant_service = None
|
||||
rag_manager = None
|
||||
if multi_tenant_enabled:
|
||||
try:
|
||||
# Create TenantService - will use rag.full_docs for db access
|
||||
# The db pool is initialized in the lifespan context
|
||||
tenant_service = TenantService(rag.full_docs)
|
||||
|
||||
# Initialize tenant RAG manager with template RAG
|
||||
rag_manager = TenantRAGManager(
|
||||
base_working_dir=args.working_dir,
|
||||
tenant_service=tenant_service,
|
||||
template_rag=rag,
|
||||
max_cached_instances=100,
|
||||
)
|
||||
|
||||
# Store in app.state for use by dependencies
|
||||
app.state.tenant_service = tenant_service
|
||||
app.state.rag_manager = rag_manager
|
||||
|
||||
logger.info("Multi-tenant mode enabled - tenant components initialized")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to initialize multi-tenant components: {e}")
|
||||
raise
|
||||
|
||||
# Add routes (rag_manager is passed for multi-tenant support, None for single-tenant)
|
||||
app.include_router(
|
||||
create_document_routes(
|
||||
rag,
|
||||
doc_manager,
|
||||
api_key,
|
||||
rag_manager=rag_manager,
|
||||
)
|
||||
)
|
||||
app.include_router(create_query_routes(rag, api_key, args.top_k))
|
||||
app.include_router(create_graph_routes(rag, api_key))
|
||||
app.include_router(create_query_routes(rag, api_key, args.top_k, rag_manager=rag_manager))
|
||||
app.include_router(create_graph_routes(rag, api_key, rag_manager=rag_manager))
|
||||
|
||||
# Add Ollama API routes
|
||||
ollama_api = OllamaAPI(rag, top_k=args.top_k, api_key=api_key)
|
||||
app.include_router(ollama_api.router, prefix="/api")
|
||||
|
||||
# Add tenant routes if multi-tenant mode is enabled
|
||||
if multi_tenant_enabled and tenant_service:
|
||||
app.include_router(create_tenant_routes(tenant_service))
|
||||
logger.info("Multi-tenant routes registered")
|
||||
|
||||
# Custom Swagger UI endpoint for offline support
|
||||
@app.get("/docs", include_in_schema=False)
|
||||
async def custom_swagger_ui_html():
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ import shutil
|
|||
import traceback
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Any, Literal
|
||||
from typing import Dict, List, Optional, Any, Literal, TYPE_CHECKING
|
||||
from io import BytesIO
|
||||
from fastapi import (
|
||||
APIRouter,
|
||||
|
|
@ -32,6 +32,10 @@ from lightrag.utils import (
|
|||
from lightrag.api.utils_api import get_combined_auth_dependency
|
||||
from ..config import global_args
|
||||
|
||||
# Type checking import to avoid circular dependencies
|
||||
if TYPE_CHECKING:
|
||||
from lightrag.tenant_rag_manager import TenantRAGManager
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def _is_docling_available() -> bool:
|
||||
|
|
@ -2035,10 +2039,44 @@ async def background_delete_documents(
|
|||
|
||||
|
||||
def create_document_routes(
|
||||
rag: LightRAG, doc_manager: DocumentManager, api_key: Optional[str] = None
|
||||
rag: LightRAG,
|
||||
doc_manager: DocumentManager,
|
||||
api_key: Optional[str] = None,
|
||||
rag_manager: Optional["TenantRAGManager"] = None,
|
||||
):
|
||||
"""Create document routes with optional multi-tenant support.
|
||||
|
||||
Args:
|
||||
rag: Default/global LightRAG instance
|
||||
doc_manager: Document manager for file operations
|
||||
api_key: Optional API key for authentication
|
||||
rag_manager: Optional TenantRAGManager for multi-tenant mode
|
||||
"""
|
||||
# Import here to avoid circular dependencies
|
||||
from lightrag.api.dependencies import get_tenant_context_optional
|
||||
from lightrag.models.tenant import TenantContext
|
||||
|
||||
# Create combined auth dependency for document routes
|
||||
combined_auth = get_combined_auth_dependency(api_key)
|
||||
|
||||
async def get_tenant_rag(
|
||||
tenant_context: Optional[TenantContext] = Depends(get_tenant_context_optional)
|
||||
) -> LightRAG:
|
||||
"""Dependency to get tenant-specific RAG instance for document operations.
|
||||
|
||||
In multi-tenant mode (when rag_manager is provided), returns tenant-specific RAG.
|
||||
Otherwise, falls back to the global RAG instance.
|
||||
"""
|
||||
if rag_manager and tenant_context and tenant_context.tenant_id and tenant_context.kb_id:
|
||||
try:
|
||||
return await rag_manager.get_rag_instance(
|
||||
tenant_context.tenant_id,
|
||||
tenant_context.kb_id,
|
||||
tenant_context.user_id
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to get tenant RAG instance: {e}, falling back to global")
|
||||
return rag
|
||||
|
||||
@router.post(
|
||||
"/scan", response_model=ScanResponse, dependencies=[Depends(combined_auth)]
|
||||
|
|
@ -2500,12 +2538,16 @@ def create_document_routes(
|
|||
dependencies=[Depends(combined_auth)],
|
||||
response_model=PipelineStatusResponse,
|
||||
)
|
||||
async def get_pipeline_status() -> PipelineStatusResponse:
|
||||
async def get_pipeline_status(
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag)
|
||||
) -> PipelineStatusResponse:
|
||||
"""
|
||||
Get the current status of the document indexing pipeline.
|
||||
|
||||
This endpoint returns information about the current state of the document processing pipeline,
|
||||
including the processing status, progress information, and history messages.
|
||||
|
||||
In multi-tenant mode, returns pipeline status for the current tenant/KB context.
|
||||
|
||||
Returns:
|
||||
PipelineStatusResponse: A response object containing:
|
||||
|
|
@ -2531,15 +2573,18 @@ def create_document_routes(
|
|||
get_all_update_flags_status,
|
||||
)
|
||||
|
||||
# Use tenant-specific workspace for pipeline status
|
||||
workspace = tenant_rag.workspace
|
||||
|
||||
pipeline_status = await get_namespace_data(
|
||||
"pipeline_status", workspace=rag.workspace
|
||||
"pipeline_status", workspace=workspace
|
||||
)
|
||||
pipeline_status_lock = get_namespace_lock(
|
||||
"pipeline_status", workspace=rag.workspace
|
||||
"pipeline_status", workspace=workspace
|
||||
)
|
||||
|
||||
# Get update flags status for all namespaces
|
||||
update_status = await get_all_update_flags_status(workspace=rag.workspace)
|
||||
update_status = await get_all_update_flags_status(workspace=workspace)
|
||||
|
||||
# Convert MutableBoolean objects to regular boolean values
|
||||
processed_update_status = {}
|
||||
|
|
@ -2973,6 +3018,7 @@ def create_document_routes(
|
|||
)
|
||||
async def get_documents_paginated(
|
||||
request: DocumentsRequest,
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
) -> PaginatedDocsResponse:
|
||||
"""
|
||||
Get documents with pagination support.
|
||||
|
|
@ -2980,6 +3026,8 @@ def create_document_routes(
|
|||
This endpoint retrieves documents with pagination, filtering, and sorting capabilities.
|
||||
It provides better performance for large document collections by loading only the
|
||||
requested page of data.
|
||||
|
||||
In multi-tenant mode, returns documents only for the current tenant/KB context.
|
||||
|
||||
Args:
|
||||
request (DocumentsRequest): The request body containing pagination parameters
|
||||
|
|
@ -2995,14 +3043,15 @@ def create_document_routes(
|
|||
"""
|
||||
try:
|
||||
# Get paginated documents and status counts in parallel
|
||||
docs_task = rag.doc_status.get_docs_paginated(
|
||||
# Use tenant-specific RAG for document operations
|
||||
docs_task = tenant_rag.doc_status.get_docs_paginated(
|
||||
status_filter=request.status_filter,
|
||||
page=request.page,
|
||||
page_size=request.page_size,
|
||||
sort_field=request.sort_field,
|
||||
sort_direction=request.sort_direction,
|
||||
)
|
||||
status_counts_task = rag.doc_status.get_all_status_counts()
|
||||
status_counts_task = tenant_rag.doc_status.get_all_status_counts()
|
||||
|
||||
# Execute both queries in parallel
|
||||
(documents_with_ids, total_count), status_counts = await asyncio.gather(
|
||||
|
|
@ -3058,12 +3107,16 @@ def create_document_routes(
|
|||
response_model=StatusCountsResponse,
|
||||
dependencies=[Depends(combined_auth)],
|
||||
)
|
||||
async def get_document_status_counts() -> StatusCountsResponse:
|
||||
async def get_document_status_counts(
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag)
|
||||
) -> StatusCountsResponse:
|
||||
"""
|
||||
Get counts of documents by status.
|
||||
|
||||
This endpoint retrieves the count of documents in each processing status
|
||||
(PENDING, PROCESSING, PROCESSED, FAILED) for all documents in the system.
|
||||
|
||||
In multi-tenant mode, returns counts only for the current tenant/KB context.
|
||||
|
||||
Returns:
|
||||
StatusCountsResponse: A response object containing status counts
|
||||
|
|
@ -3072,7 +3125,8 @@ def create_document_routes(
|
|||
HTTPException: If an error occurs while retrieving status counts (500).
|
||||
"""
|
||||
try:
|
||||
status_counts = await rag.doc_status.get_all_status_counts()
|
||||
# Use tenant-specific RAG for document status counts
|
||||
status_counts = await tenant_rag.doc_status.get_all_status_counts()
|
||||
return StatusCountsResponse(status_counts=status_counts)
|
||||
|
||||
except Exception as e:
|
||||
|
|
|
|||
|
|
@ -2,14 +2,19 @@
|
|||
This module contains all graph-related routes for the LightRAG API.
|
||||
"""
|
||||
|
||||
from typing import Optional, Dict, Any
|
||||
from typing import Optional, Dict, Any, TYPE_CHECKING
|
||||
import traceback
|
||||
from fastapi import APIRouter, Depends, Query, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from lightrag import LightRAG
|
||||
from lightrag.utils import logger
|
||||
from ..utils_api import get_combined_auth_dependency
|
||||
|
||||
# Type checking import to avoid circular dependencies
|
||||
if TYPE_CHECKING:
|
||||
from lightrag.tenant_rag_manager import TenantRAGManager
|
||||
|
||||
router = APIRouter(tags=["graph"])
|
||||
|
||||
|
||||
|
|
@ -86,11 +91,47 @@ class RelationCreateRequest(BaseModel):
|
|||
)
|
||||
|
||||
|
||||
def create_graph_routes(rag, api_key: Optional[str] = None):
|
||||
def create_graph_routes(
|
||||
rag: LightRAG,
|
||||
api_key: Optional[str] = None,
|
||||
rag_manager: Optional["TenantRAGManager"] = None,
|
||||
):
|
||||
"""Create graph routes with optional multi-tenant support.
|
||||
|
||||
Args:
|
||||
rag: Default/global LightRAG instance
|
||||
api_key: Optional API key for authentication
|
||||
rag_manager: Optional TenantRAGManager for multi-tenant mode
|
||||
"""
|
||||
# Import here to avoid circular dependencies
|
||||
from lightrag.api.dependencies import get_tenant_context_optional
|
||||
from lightrag.models.tenant import TenantContext
|
||||
|
||||
combined_auth = get_combined_auth_dependency(api_key)
|
||||
|
||||
async def get_tenant_rag(
|
||||
tenant_context: Optional[TenantContext] = Depends(get_tenant_context_optional)
|
||||
) -> LightRAG:
|
||||
"""Dependency to get tenant-specific RAG instance for graph operations.
|
||||
|
||||
In multi-tenant mode (when rag_manager is provided), returns tenant-specific RAG.
|
||||
Otherwise, falls back to the global RAG instance.
|
||||
"""
|
||||
if rag_manager and tenant_context and tenant_context.tenant_id and tenant_context.kb_id:
|
||||
try:
|
||||
return await rag_manager.get_rag_instance(
|
||||
tenant_context.tenant_id,
|
||||
tenant_context.kb_id,
|
||||
tenant_context.user_id
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to get tenant RAG instance: {e}, falling back to global")
|
||||
return rag
|
||||
|
||||
@router.get("/graph/label/list", dependencies=[Depends(combined_auth)])
|
||||
async def get_graph_labels():
|
||||
async def get_graph_labels(
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag)
|
||||
):
|
||||
"""
|
||||
Get all graph labels
|
||||
|
||||
|
|
@ -98,7 +139,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
List[str]: List of graph labels
|
||||
"""
|
||||
try:
|
||||
return await rag.get_graph_labels()
|
||||
return await tenant_rag.get_graph_labels()
|
||||
except Exception as e:
|
||||
logger.error(f"Error getting graph labels: {str(e)}")
|
||||
logger.error(traceback.format_exc())
|
||||
|
|
@ -108,6 +149,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
|
||||
@router.get("/graph/label/popular", dependencies=[Depends(combined_auth)])
|
||||
async def get_popular_labels(
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
limit: int = Query(
|
||||
300, description="Maximum number of popular labels to return", ge=1, le=1000
|
||||
),
|
||||
|
|
@ -122,7 +164,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
List[str]: List of popular labels sorted by degree (highest first)
|
||||
"""
|
||||
try:
|
||||
return await rag.chunk_entity_relation_graph.get_popular_labels(limit)
|
||||
return await tenant_rag.chunk_entity_relation_graph.get_popular_labels(limit)
|
||||
except Exception as e:
|
||||
logger.error(f"Error getting popular labels: {str(e)}")
|
||||
logger.error(traceback.format_exc())
|
||||
|
|
@ -132,6 +174,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
|
||||
@router.get("/graph/label/search", dependencies=[Depends(combined_auth)])
|
||||
async def search_labels(
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
q: str = Query(..., description="Search query string"),
|
||||
limit: int = Query(
|
||||
50, description="Maximum number of search results to return", ge=1, le=100
|
||||
|
|
@ -148,7 +191,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
List[str]: List of matching labels sorted by relevance
|
||||
"""
|
||||
try:
|
||||
return await rag.chunk_entity_relation_graph.search_labels(q, limit)
|
||||
return await tenant_rag.chunk_entity_relation_graph.search_labels(q, limit)
|
||||
except Exception as e:
|
||||
logger.error(f"Error searching labels with query '{q}': {str(e)}")
|
||||
logger.error(traceback.format_exc())
|
||||
|
|
@ -158,6 +201,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
|
||||
@router.get("/graphs", dependencies=[Depends(combined_auth)])
|
||||
async def get_knowledge_graph(
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
label: str = Query(..., description="Label to get knowledge graph for"),
|
||||
max_depth: int = Query(3, description="Maximum depth of graph", ge=1),
|
||||
max_nodes: int = Query(1000, description="Maximum nodes to return", ge=1),
|
||||
|
|
@ -182,7 +226,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
f"get_knowledge_graph called with label: '{label}' (length: {len(label)}, repr: {repr(label)})"
|
||||
)
|
||||
|
||||
return await rag.get_knowledge_graph(
|
||||
return await tenant_rag.get_knowledge_graph(
|
||||
node_label=label,
|
||||
max_depth=max_depth,
|
||||
max_nodes=max_nodes,
|
||||
|
|
@ -196,6 +240,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
|
||||
@router.get("/graph/entity/exists", dependencies=[Depends(combined_auth)])
|
||||
async def check_entity_exists(
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
name: str = Query(..., description="Entity name to check"),
|
||||
):
|
||||
"""
|
||||
|
|
@ -208,7 +253,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
Dict[str, bool]: Dictionary with 'exists' key indicating if entity exists
|
||||
"""
|
||||
try:
|
||||
exists = await rag.chunk_entity_relation_graph.has_node(name)
|
||||
exists = await tenant_rag.chunk_entity_relation_graph.has_node(name)
|
||||
return {"exists": exists}
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking entity existence for '{name}': {str(e)}")
|
||||
|
|
@ -218,7 +263,10 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
)
|
||||
|
||||
@router.post("/graph/entity/edit", dependencies=[Depends(combined_auth)])
|
||||
async def update_entity(request: EntityUpdateRequest):
|
||||
async def update_entity(
|
||||
request: EntityUpdateRequest,
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
):
|
||||
"""
|
||||
Update an entity's properties in the knowledge graph
|
||||
|
||||
|
|
@ -263,7 +311,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
}
|
||||
"""
|
||||
try:
|
||||
result = await rag.aedit_entity(
|
||||
result = await tenant_rag.aedit_entity(
|
||||
entity_name=request.entity_name,
|
||||
updated_data=request.updated_data,
|
||||
allow_rename=request.allow_rename,
|
||||
|
|
@ -287,7 +335,10 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
)
|
||||
|
||||
@router.post("/graph/relation/edit", dependencies=[Depends(combined_auth)])
|
||||
async def update_relation(request: RelationUpdateRequest):
|
||||
async def update_relation(
|
||||
request: RelationUpdateRequest,
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
):
|
||||
"""Update a relation's properties in the knowledge graph
|
||||
|
||||
Args:
|
||||
|
|
@ -297,7 +348,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
Dict: Updated relation information
|
||||
"""
|
||||
try:
|
||||
result = await rag.aedit_relation(
|
||||
result = await tenant_rag.aedit_relation(
|
||||
source_entity=request.source_id,
|
||||
target_entity=request.target_id,
|
||||
updated_data=request.updated_data,
|
||||
|
|
@ -322,7 +373,10 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
)
|
||||
|
||||
@router.post("/graph/entity/create", dependencies=[Depends(combined_auth)])
|
||||
async def create_entity(request: EntityCreateRequest):
|
||||
async def create_entity(
|
||||
request: EntityCreateRequest,
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
):
|
||||
"""
|
||||
Create a new entity in the knowledge graph
|
||||
|
||||
|
|
@ -372,7 +426,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
# - Vector embedding creation in entities_vdb
|
||||
# - Metadata population and defaults
|
||||
# - Index consistency via _edit_entity_done
|
||||
result = await rag.acreate_entity(
|
||||
result = await tenant_rag.acreate_entity(
|
||||
entity_name=request.entity_name,
|
||||
entity_data=request.entity_data,
|
||||
)
|
||||
|
|
@ -395,7 +449,10 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
)
|
||||
|
||||
@router.post("/graph/relation/create", dependencies=[Depends(combined_auth)])
|
||||
async def create_relation(request: RelationCreateRequest):
|
||||
async def create_relation(
|
||||
request: RelationCreateRequest,
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
):
|
||||
"""
|
||||
Create a new relationship between two entities in the knowledge graph
|
||||
|
||||
|
|
@ -458,7 +515,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
# - Duplicate relation checks
|
||||
# - Vector embedding creation in relationships_vdb
|
||||
# - Index consistency via _edit_relation_done
|
||||
result = await rag.acreate_relation(
|
||||
result = await tenant_rag.acreate_relation(
|
||||
source_entity=request.source_entity,
|
||||
target_entity=request.target_entity,
|
||||
relation_data=request.relation_data,
|
||||
|
|
@ -484,7 +541,10 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
)
|
||||
|
||||
@router.post("/graph/entities/merge", dependencies=[Depends(combined_auth)])
|
||||
async def merge_entities(request: EntityMergeRequest):
|
||||
async def merge_entities(
|
||||
request: EntityMergeRequest,
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
):
|
||||
"""
|
||||
Merge multiple entities into a single entity, preserving all relationships
|
||||
|
||||
|
|
@ -541,7 +601,7 @@ def create_graph_routes(rag, api_key: Optional[str] = None):
|
|||
- This operation cannot be undone, so verify entity names before merging
|
||||
"""
|
||||
try:
|
||||
result = await rag.amerge_entities(
|
||||
result = await tenant_rag.amerge_entities(
|
||||
source_entities=request.entities_to_change,
|
||||
target_entity=request.entity_to_change_into,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -4,15 +4,20 @@ This module contains all query-related routes for the LightRAG API.
|
|||
|
||||
import json
|
||||
import logging
|
||||
from typing import Any, Dict, List, Literal, Optional
|
||||
from typing import Any, Dict, List, Literal, Optional, TYPE_CHECKING
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from lightrag import LightRAG
|
||||
from lightrag.base import QueryParam
|
||||
from lightrag.api.utils_api import get_combined_auth_dependency
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from ascii_colors import trace_exception
|
||||
|
||||
# Type checking import to avoid circular dependencies
|
||||
if TYPE_CHECKING:
|
||||
from lightrag.tenant_rag_manager import TenantRAGManager
|
||||
|
||||
router = APIRouter(tags=["query"])
|
||||
|
||||
|
||||
|
|
@ -182,8 +187,44 @@ class StreamChunkResponse(BaseModel):
|
|||
)
|
||||
|
||||
|
||||
def create_query_routes(rag, api_key: Optional[str] = None, top_k: int = 60):
|
||||
def create_query_routes(
|
||||
rag: LightRAG,
|
||||
api_key: Optional[str] = None,
|
||||
top_k: int = 60,
|
||||
rag_manager: Optional["TenantRAGManager"] = None,
|
||||
):
|
||||
"""Create query routes with optional multi-tenant support.
|
||||
|
||||
Args:
|
||||
rag: Default/global LightRAG instance
|
||||
api_key: Optional API key for authentication
|
||||
top_k: Default top_k value for queries
|
||||
rag_manager: Optional TenantRAGManager for multi-tenant mode
|
||||
"""
|
||||
# Import here to avoid circular dependencies
|
||||
from lightrag.api.dependencies import get_tenant_context_optional
|
||||
from lightrag.models.tenant import TenantContext
|
||||
|
||||
combined_auth = get_combined_auth_dependency(api_key)
|
||||
|
||||
async def get_tenant_rag(
|
||||
tenant_context: Optional[TenantContext] = Depends(get_tenant_context_optional)
|
||||
) -> LightRAG:
|
||||
"""Dependency to get tenant-specific RAG instance for query operations.
|
||||
|
||||
In multi-tenant mode (when rag_manager is provided), returns tenant-specific RAG.
|
||||
Otherwise, falls back to the global RAG instance.
|
||||
"""
|
||||
if rag_manager and tenant_context and tenant_context.tenant_id and tenant_context.kb_id:
|
||||
try:
|
||||
return await rag_manager.get_rag_instance(
|
||||
tenant_context.tenant_id,
|
||||
tenant_context.kb_id,
|
||||
tenant_context.user_id
|
||||
)
|
||||
except Exception as e:
|
||||
logging.warning(f"Failed to get tenant RAG instance: {e}, falling back to global")
|
||||
return rag
|
||||
|
||||
@router.post(
|
||||
"/query",
|
||||
|
|
@ -304,7 +345,10 @@ def create_query_routes(rag, api_key: Optional[str] = None, top_k: int = 60):
|
|||
},
|
||||
},
|
||||
)
|
||||
async def query_text(request: QueryRequest):
|
||||
async def query_text(
|
||||
request: QueryRequest,
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
):
|
||||
"""
|
||||
Comprehensive RAG query endpoint with non-streaming response. Parameter "stream" is ignored.
|
||||
|
||||
|
|
@ -391,7 +435,7 @@ def create_query_routes(rag, api_key: Optional[str] = None, top_k: int = 60):
|
|||
param.stream = False
|
||||
|
||||
# Unified approach: always use aquery_llm for both cases
|
||||
result = await rag.aquery_llm(request.query, param=param)
|
||||
result = await tenant_rag.aquery_llm(request.query, param=param)
|
||||
|
||||
# Extract LLM response and references from unified result
|
||||
llm_response = result.get("llm_response", {})
|
||||
|
|
@ -508,7 +552,10 @@ def create_query_routes(rag, api_key: Optional[str] = None, top_k: int = 60):
|
|||
},
|
||||
},
|
||||
)
|
||||
async def query_text_stream(request: QueryRequest):
|
||||
async def query_text_stream(
|
||||
request: QueryRequest,
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
):
|
||||
"""
|
||||
Advanced RAG query endpoint with flexible streaming response.
|
||||
|
||||
|
|
@ -643,7 +690,7 @@ def create_query_routes(rag, api_key: Optional[str] = None, top_k: int = 60):
|
|||
from fastapi.responses import StreamingResponse
|
||||
|
||||
# Unified approach: always use aquery_llm for all cases
|
||||
result = await rag.aquery_llm(request.query, param=param)
|
||||
result = await tenant_rag.aquery_llm(request.query, param=param)
|
||||
|
||||
async def stream_generator():
|
||||
# Extract references and LLM response from unified result
|
||||
|
|
@ -987,7 +1034,10 @@ def create_query_routes(rag, api_key: Optional[str] = None, top_k: int = 60):
|
|||
},
|
||||
},
|
||||
)
|
||||
async def query_data(request: QueryRequest):
|
||||
async def query_data(
|
||||
request: QueryRequest,
|
||||
tenant_rag: LightRAG = Depends(get_tenant_rag),
|
||||
):
|
||||
"""
|
||||
Advanced data retrieval endpoint for structured RAG analysis.
|
||||
|
||||
|
|
@ -1092,7 +1142,7 @@ def create_query_routes(rag, api_key: Optional[str] = None, top_k: int = 60):
|
|||
"""
|
||||
try:
|
||||
param = request.to_query_params(False) # No streaming for data endpoint
|
||||
response = await rag.aquery_data(request.query, param=param)
|
||||
response = await tenant_rag.aquery_data(request.query, param=param)
|
||||
|
||||
# aquery_data returns the new format with status, message, data, and metadata
|
||||
if isinstance(response, dict):
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
"""Service for managing tenants and knowledge bases."""
|
||||
|
||||
from typing import Optional, List, Dict, Any
|
||||
from typing import Optional, Dict, Any
|
||||
import logging
|
||||
from datetime import datetime
|
||||
|
||||
|
|
@ -217,7 +217,7 @@ class TenantService:
|
|||
# Function might not exist if migration hasn't run - use legacy fallback
|
||||
error_msg = str(e)
|
||||
if "has_tenant_access" in error_msg and "does not exist" in error_msg:
|
||||
logger.debug(f"has_tenant_access function not found, using legacy access check")
|
||||
logger.debug("has_tenant_access function not found, using legacy access check")
|
||||
else:
|
||||
logger.warning(f"Error checking user access: {e}")
|
||||
# Fall through to legacy check
|
||||
|
|
@ -744,7 +744,12 @@ class TenantService:
|
|||
return {"items": [], "total": 0}
|
||||
|
||||
async def delete_tenant(self, tenant_id: str) -> bool:
|
||||
"""Delete a tenant.
|
||||
"""Delete a tenant and all associated data.
|
||||
|
||||
This method performs cascade delete:
|
||||
1. Deletes all knowledge bases (which cascade delete their LIGHTRAG_* data)
|
||||
2. Deletes user-tenant memberships
|
||||
3. Deletes tenant metadata from PostgreSQL and KV storage
|
||||
|
||||
Args:
|
||||
tenant_id: Tenant identifier
|
||||
|
|
@ -756,18 +761,39 @@ class TenantService:
|
|||
if not tenant:
|
||||
return False
|
||||
|
||||
# Delete all KBs associated with tenant
|
||||
# Delete all KBs associated with tenant (includes cascade delete of LIGHTRAG_* data)
|
||||
kbs_result = await self.list_knowledge_bases(tenant_id)
|
||||
kbs_list = kbs_result.get("items", [])
|
||||
for kb in kbs_list:
|
||||
await self.delete_knowledge_base(tenant_id, kb.kb_id)
|
||||
|
||||
# Delete tenant
|
||||
# Delete user-tenant memberships from PostgreSQL
|
||||
if hasattr(self.kv_storage, 'db') and self.kv_storage.db is not None:
|
||||
try:
|
||||
await self.kv_storage.db.execute(
|
||||
"DELETE FROM user_tenant_memberships WHERE tenant_id = $1",
|
||||
[tenant_id]
|
||||
)
|
||||
logger.debug(f"Deleted user memberships for tenant {tenant_id}")
|
||||
except Exception as e:
|
||||
logger.debug(f"Could not delete user memberships: {e}")
|
||||
|
||||
# Delete from tenants table (FK cascade should handle knowledge_bases)
|
||||
try:
|
||||
await self.kv_storage.db.execute(
|
||||
"DELETE FROM tenants WHERE tenant_id = $1",
|
||||
[tenant_id]
|
||||
)
|
||||
logger.debug(f"Deleted tenant {tenant_id} from PostgreSQL")
|
||||
except Exception as e:
|
||||
logger.debug(f"Could not delete from tenants table: {e}")
|
||||
|
||||
# Delete tenant metadata from KV storage
|
||||
await self.kv_storage.delete(
|
||||
[f"{self.tenant_namespace}:{tenant_id}"]
|
||||
)
|
||||
|
||||
logger.info(f"Deleted tenant: {tenant_id}")
|
||||
logger.info(f"Deleted tenant: {tenant_id} (with cascade delete)")
|
||||
return True
|
||||
|
||||
async def create_knowledge_base(
|
||||
|
|
@ -1002,7 +1028,12 @@ class TenantService:
|
|||
tenant_id: str,
|
||||
kb_id: str,
|
||||
) -> bool:
|
||||
"""Delete a knowledge base.
|
||||
"""Delete a knowledge base and all associated data.
|
||||
|
||||
This method performs cascade delete:
|
||||
1. Deletes all LIGHTRAG_* table data for this workspace
|
||||
2. Deletes KB metadata from KV storage
|
||||
3. Updates tenant KB count
|
||||
|
||||
Args:
|
||||
tenant_id: Parent tenant ID
|
||||
|
|
@ -1015,18 +1046,67 @@ class TenantService:
|
|||
if not kb:
|
||||
return False
|
||||
|
||||
# Delete KB
|
||||
# Cascade delete: Clean up LIGHTRAG_* tables for this workspace
|
||||
workspace = f"{tenant_id}:{kb_id}"
|
||||
if hasattr(self.kv_storage, 'db') and self.kv_storage.db is not None:
|
||||
try:
|
||||
# List of all LIGHTRAG tables that use workspace
|
||||
lightrag_tables = [
|
||||
"LIGHTRAG_DOC_FULL",
|
||||
"LIGHTRAG_DOC_CHUNKS",
|
||||
"LIGHTRAG_DOC_STATUS",
|
||||
"LIGHTRAG_VDB_CHUNKS",
|
||||
"LIGHTRAG_VDB_ENTITY",
|
||||
"LIGHTRAG_VDB_RELATION",
|
||||
"LIGHTRAG_LLM_CACHE",
|
||||
"LIGHTRAG_FULL_ENTITIES",
|
||||
"LIGHTRAG_FULL_RELATIONS",
|
||||
"LIGHTRAG_ENTITY_CHUNKS",
|
||||
"LIGHTRAG_RELATION_CHUNKS",
|
||||
]
|
||||
|
||||
total_deleted = 0
|
||||
for table in lightrag_tables:
|
||||
try:
|
||||
result = await self.kv_storage.db.execute(
|
||||
f"DELETE FROM {table} WHERE workspace = $1",
|
||||
[workspace]
|
||||
)
|
||||
# Log if rows were deleted (result may be row count or None)
|
||||
if result:
|
||||
logger.debug(f"Deleted rows from {table} for workspace {workspace}")
|
||||
total_deleted += 1
|
||||
except Exception as table_error:
|
||||
# Table might not exist, log and continue
|
||||
logger.debug(f"Could not delete from {table}: {table_error}")
|
||||
|
||||
logger.info(f"Cascade delete: cleaned up LIGHTRAG tables for workspace {workspace}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Error during cascade delete for KB {kb_id}: {e}")
|
||||
# Continue with KB deletion even if cascade fails
|
||||
|
||||
# Delete KB metadata from KV storage
|
||||
await self.kv_storage.delete(
|
||||
[f"{self.kb_namespace}:{tenant_id}:{kb_id}"]
|
||||
)
|
||||
|
||||
# Delete KB from knowledge_bases table if using PostgreSQL
|
||||
if hasattr(self.kv_storage, 'db') and self.kv_storage.db is not None:
|
||||
try:
|
||||
await self.kv_storage.db.execute(
|
||||
"DELETE FROM knowledge_bases WHERE tenant_id = $1 AND kb_id = $2",
|
||||
[tenant_id, kb_id]
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"Could not delete from knowledge_bases table: {e}")
|
||||
|
||||
# Update tenant KB count
|
||||
tenant = await self.get_tenant(tenant_id)
|
||||
if tenant:
|
||||
tenant.kb_count = max(0, tenant.kb_count - 1)
|
||||
await self.update_tenant(tenant_id, kb_count=tenant.kb_count)
|
||||
|
||||
logger.info(f"Deleted KB: {kb_id} for tenant {tenant_id}")
|
||||
logger.info(f"Deleted KB: {kb_id} for tenant {tenant_id} (with cascade delete)")
|
||||
return True
|
||||
|
||||
def _deserialize_tenant(self, data: Dict[str, Any]) -> Tenant:
|
||||
|
|
@ -1049,7 +1129,7 @@ class TenantService:
|
|||
logger.warning(f"Deserializing tenant with missing ID. Data keys: {list(data.keys())}")
|
||||
|
||||
config_data = data.get("config", {})
|
||||
quota_data = data.get("quota", {})
|
||||
data.get("quota", {})
|
||||
|
||||
config = TenantConfig(
|
||||
llm_model=config_data.get("llm_model", "gpt-4o-mini"),
|
||||
|
|
|
|||
|
|
@ -1,49 +1,107 @@
|
|||
import { defineConfig } from 'vite'
|
||||
import { defineConfig, loadEnv } from 'vite'
|
||||
import path from 'path'
|
||||
import { webuiPrefix } from '@/lib/constants'
|
||||
import react from '@vitejs/plugin-react-swc'
|
||||
import tailwindcss from '@tailwindcss/vite'
|
||||
|
||||
// WebUI base path - must match the value in src/lib/constants.ts
|
||||
const webuiPrefix = '/webui/'
|
||||
|
||||
// https://vite.dev/config/
|
||||
export default defineConfig({
|
||||
plugins: [react(), tailwindcss()],
|
||||
resolve: {
|
||||
alias: {
|
||||
'@': path.resolve(__dirname, './src')
|
||||
}
|
||||
},
|
||||
// base: import.meta.env.VITE_BASE_URL || '/webui/',
|
||||
base: webuiPrefix,
|
||||
build: {
|
||||
outDir: path.resolve(__dirname, '../lightrag/api/webui'),
|
||||
emptyOutDir: true,
|
||||
chunkSizeWarningLimit: 3800,
|
||||
rollupOptions: {
|
||||
// Let Vite handle chunking automatically to avoid circular dependency issues
|
||||
output: {
|
||||
// Ensure consistent chunk naming format
|
||||
chunkFileNames: 'assets/[name]-[hash].js',
|
||||
// Entry file naming format
|
||||
entryFileNames: 'assets/[name]-[hash].js',
|
||||
// Asset file naming format
|
||||
assetFileNames: 'assets/[name]-[hash].[ext]'
|
||||
export default defineConfig(({ mode }) => {
|
||||
// Load env file based on `mode` in the current working directory.
|
||||
const env = loadEnv(mode, process.cwd(), '')
|
||||
|
||||
// Backend URL for API proxy (default to local dev server)
|
||||
// Use 127.0.0.1 instead of localhost to avoid IPv6 resolution issues
|
||||
const backendUrl = env.VITE_BACKEND_URL || 'http://127.0.0.1:9621'
|
||||
|
||||
return {
|
||||
plugins: [react(), tailwindcss()],
|
||||
resolve: {
|
||||
alias: {
|
||||
'@': path.resolve(__dirname, './src')
|
||||
}
|
||||
},
|
||||
// base: import.meta.env.VITE_BASE_URL || '/webui/',
|
||||
base: webuiPrefix,
|
||||
build: {
|
||||
outDir: path.resolve(__dirname, '../lightrag/api/webui'),
|
||||
emptyOutDir: true,
|
||||
chunkSizeWarningLimit: 3800,
|
||||
rollupOptions: {
|
||||
// Let Vite handle chunking automatically to avoid circular dependency issues
|
||||
output: {
|
||||
// Ensure consistent chunk naming format
|
||||
chunkFileNames: 'assets/[name]-[hash].js',
|
||||
// Entry file naming format
|
||||
entryFileNames: 'assets/[name]-[hash].js',
|
||||
// Asset file naming format
|
||||
assetFileNames: 'assets/[name]-[hash].[ext]'
|
||||
}
|
||||
}
|
||||
},
|
||||
server: {
|
||||
// Proxy all API routes to the backend during development
|
||||
proxy: {
|
||||
// API v1 routes (tenant management, knowledge bases, etc.)
|
||||
'/api/v1': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
// Legacy API routes (chat, generate, tags, etc.)
|
||||
'/api': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
// Document operations
|
||||
'/documents': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
// Query operations
|
||||
'/query': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
// Graph operations
|
||||
'/graph': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
// Retrieval operations
|
||||
'/retrieval': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
// Health check
|
||||
'/health': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
// Authentication status
|
||||
'/auth-status': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
// OpenAPI docs
|
||||
'/docs': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
'/redoc': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
'/openapi.json': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
// Static assets for Swagger UI
|
||||
'/static': {
|
||||
target: backendUrl,
|
||||
changeOrigin: true,
|
||||
},
|
||||
}
|
||||
}
|
||||
},
|
||||
server: {
|
||||
proxy: import.meta.env.VITE_API_PROXY === 'true' && import.meta.env.VITE_API_ENDPOINTS ?
|
||||
Object.fromEntries(
|
||||
import.meta.env.VITE_API_ENDPOINTS.split(',').map(endpoint => [
|
||||
endpoint,
|
||||
{
|
||||
target: import.meta.env.VITE_BACKEND_URL || 'http://localhost:9621',
|
||||
changeOrigin: true,
|
||||
rewrite: endpoint === '/api' ?
|
||||
(path) => path.replace(/^\/api/, '') :
|
||||
endpoint === '/docs' || endpoint === '/redoc' || endpoint === '/openapi.json' ?
|
||||
(path) => path : undefined
|
||||
}
|
||||
])
|
||||
) : {}
|
||||
}
|
||||
})
|
||||
|
|
|
|||
40
logs/2024-12-05-01-30-beastmode-audit-log.md
Normal file
40
logs/2024-12-05-01-30-beastmode-audit-log.md
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
# Task Log: Multi-Tenant vs Workspace Audit
|
||||
|
||||
**Date:** 2024-12-05 01:30 UTC
|
||||
**Mode:** Beastmode
|
||||
**Task:** Audit Multi-Tenant vs Workspace redundancy
|
||||
|
||||
## Actions
|
||||
|
||||
1. Fetched upstream LightRAG README to understand workspace feature
|
||||
2. Analyzed core `lightrag/lightrag.py` workspace parameter usage
|
||||
3. Analyzed `lightrag/kg/postgres_impl.py` for table schemas with workspace
|
||||
4. Analyzed `lightrag/services/tenant_service.py` for tenant management
|
||||
5. Analyzed `lightrag/tenant_rag_manager.py` for RAG instance management
|
||||
6. Analyzed `lightrag/models/tenant.py` for data models
|
||||
7. Analyzed `lightrag/security.py` for workspace/tenant validation
|
||||
8. Analyzed `starter/init-postgres.sql` for database schema
|
||||
9. Used sequential thinking to compare architectures
|
||||
10. Created comprehensive audit report: `docs/0009-multi-tenant-vs-workspace-audit.md`
|
||||
11. Implemented cascade delete for `delete_knowledge_base()` method
|
||||
12. Implemented cascade delete for `delete_tenant()` method
|
||||
13. Fixed linting issues with ruff
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Tenant feature is NOT redundant** - it's a proper application layer on top of workspace
|
||||
- Workspace provides storage-level isolation (HOW data is separated)
|
||||
- Tenant provides application-level multi-tenancy (WHO can access WHAT)
|
||||
- Composite workspace format `{tenant_id}:{kb_id}` bridges the two layers
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Add unit tests for cascade delete functionality
|
||||
2. Consider adding DB triggers for referential integrity
|
||||
3. Update user-facing documentation with workspace naming convention
|
||||
|
||||
## Lessons/Insights
|
||||
|
||||
- LightRAG uses a layered architecture that correctly separates storage isolation from access control
|
||||
- Generated columns in PostgreSQL allow querying by tenant/KB without schema changes
|
||||
- The `TenantRAGManager` acts as a bridge, creating composite workspace identifiers
|
||||
54
logs/2024-12-05-02-00-beastmode-pipeline-filter-log.md
Normal file
54
logs/2024-12-05-02-00-beastmode-pipeline-filter-log.md
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
# Task Log: Pipeline Screen Tenant Filtering Fix
|
||||
|
||||
**Date**: 2024-12-05 02:00
|
||||
**Mode**: beastmode-chatmode
|
||||
**Task**: Fix pipeline screen not being filtered by tenant and KB
|
||||
|
||||
## Summary
|
||||
|
||||
Implemented multi-tenant support for document routes to ensure the pipeline screen filters documents by the current tenant and knowledge base (KB) context.
|
||||
|
||||
## Actions Performed
|
||||
|
||||
1. **Updated `document_routes.py`**:
|
||||
- Added imports for `TenantRAGManager`, `TenantContext`, `get_tenant_context_optional`
|
||||
- Modified `create_document_routes()` signature to accept optional `rag_manager` parameter
|
||||
- Created `get_tenant_rag` dependency that returns tenant-specific RAG instance when context is available
|
||||
- Updated `/pipeline_status` endpoint to use `tenant_rag` dependency for workspace-isolated pipeline status
|
||||
- Updated `/paginated` endpoint to use `tenant_rag` dependency for tenant-filtered document listing
|
||||
- Updated `/status_counts` endpoint to use `tenant_rag` dependency for tenant-filtered status counts
|
||||
|
||||
2. **Restructured `lightrag_server.py`**:
|
||||
- Moved multi-tenant component initialization (TenantRAGManager) before document routes registration
|
||||
- Modified `create_document_routes()` call to pass `rag_manager=rag_manager` parameter
|
||||
- Separated tenant routes registration from multi-tenant initialization
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- Used FastAPI's Depends() injection to get tenant-specific RAG instance
|
||||
- Pattern: `tenant_rag: LightRAG = Depends(get_tenant_rag)` for tenant-aware endpoints
|
||||
- Fallback to global `rag` when tenant context is not available (single-tenant mode compatibility)
|
||||
- `workspace = tenant_rag.workspace` contains the composite `{tenant_id}:{kb_id}` pattern for storage isolation
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `lightrag/api/routers/document_routes.py`
|
||||
- `lightrag/api/lightrag_server.py`
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Consider updating graph routes (`graph_routes.py`) for tenant-aware graph operations
|
||||
- Consider updating query routes (`query_routes.py`) for tenant-aware queries
|
||||
- Write/upload operations (upload, delete, etc.) may need similar tenant-aware treatment
|
||||
|
||||
## Lessons/Insights
|
||||
|
||||
- The document routes were using the global `rag` instance, which always used the default workspace
|
||||
- The fix pattern is: replace `rag.workspace` with `tenant_rag.workspace` where `tenant_rag` comes from the dependency
|
||||
- TenantRAGManager must be initialized before registering routes that depend on it
|
||||
- The `get_tenant_rag` dependency gracefully falls back to global RAG for backward compatibility
|
||||
|
||||
## Testing
|
||||
|
||||
- Ran `ruff check` on modified files - all checks passed
|
||||
- No TypeScript/Python errors detected in modified files
|
||||
76
logs/2025-01-06-09-30-beastmode-webui-single-multi-tenant.md
Normal file
76
logs/2025-01-06-09-30-beastmode-webui-single-multi-tenant.md
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
# Task Log: WebUI Single-Tenant/Multi-Tenant Mode Support
|
||||
|
||||
**Date**: 2025-01-06 09:30
|
||||
**Mode**: Beastmode
|
||||
|
||||
## Summary
|
||||
|
||||
Implemented single-tenant and multi-tenant mode support for the LightRAG WebUI to ensure it works correctly in both configurations.
|
||||
|
||||
## Actions Performed
|
||||
|
||||
1. **Added `LIGHTRAG_MULTI_TENANT` env var** to `lightrag/api/lightrag_server.py` to control multi-tenant mode
|
||||
2. **Updated `/auth-status` and `/health` endpoints** to include `multi_tenant_enabled` flag
|
||||
3. **Updated `LoginPage.tsx`** to auto-redirect in single-tenant mode (bypass tenant selection)
|
||||
4. **Updated `App.tsx`** to set default tenant AND KB in single-tenant mode
|
||||
5. **Updated `TenantSelector.tsx`** to skip API calls when `multiTenantEnabled=false`
|
||||
6. **Updated `SiteHeader.tsx`** to conditionally hide tenant selector in single-tenant mode
|
||||
7. **Updated `useTenantInitialization.ts`** hook to skip tenant API calls in single-tenant mode
|
||||
8. **Updated `AuthStore`** in `stores/state.ts` with `multiTenantEnabled` state and `setMultiTenantEnabled` action
|
||||
9. **Updated `lightrag.ts`** API types to include `multi_tenant_enabled` in `AuthStatusResponse`
|
||||
10. **Rebuilt WebUI** multiple times during development
|
||||
|
||||
## Key Decisions
|
||||
|
||||
1. **Default mode is single-tenant** (`LIGHTRAG_MULTI_TENANT=false`) for backward compatibility
|
||||
2. **Default tenant/KB IDs are "default"** to match API expectations
|
||||
3. **Auto-set both tenant AND KB** in single-tenant mode to avoid "KB context required" errors
|
||||
4. **Multi-tenant mode requires separate tenant API routes** to be configured (not fully implemented in current codebase)
|
||||
|
||||
## Test Results
|
||||
|
||||
### Single-Tenant Mode (Default)
|
||||
- ✅ WebUI loads without errors
|
||||
- ✅ Auto-login with free login mode
|
||||
- ✅ Documents tab works (shows empty state)
|
||||
- ✅ Knowledge Graph tab works (shows empty graph)
|
||||
- ✅ Retrieval tab works with query parameters
|
||||
- ✅ API tab loads Swagger UI (404 for docs endpoints expected)
|
||||
- ✅ No tenant selection UI shown
|
||||
- ✅ No 404 errors for tenant/KB API calls
|
||||
|
||||
### Multi-Tenant Mode
|
||||
- ⚠️ Requires tenant API routes to be configured
|
||||
- ⚠️ Shows 404 errors for `/api/v1/tenants` endpoint
|
||||
- ℹ️ Needs TenantService and tenant routes to be included in API server
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `lightrag/api/lightrag_server.py`
|
||||
- `lightrag_webui/src/App.tsx`
|
||||
- `lightrag_webui/src/features/LoginPage.tsx`
|
||||
- `lightrag_webui/src/features/SiteHeader.tsx`
|
||||
- `lightrag_webui/src/components/TenantSelector.tsx`
|
||||
- `lightrag_webui/src/hooks/useTenantInitialization.ts`
|
||||
- `lightrag_webui/src/stores/state.ts`
|
||||
- `lightrag_webui/src/api/lightrag.ts`
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. To enable full multi-tenant support:
|
||||
- Include tenant routes in API server when `LIGHTRAG_MULTI_TENANT=true`
|
||||
- Set up tenant/KB tables in PostgreSQL
|
||||
- Create default tenant and KB during initialization
|
||||
- Test tenant CRUD operations
|
||||
|
||||
2. Consider adding:
|
||||
- Tenant creation UI in WebUI
|
||||
- KB creation UI in WebUI
|
||||
- Tenant switching without page reload
|
||||
|
||||
## Lessons Learned
|
||||
|
||||
1. The env var name is `LIGHTRAG_MULTI_TENANT`, not `ENABLE_MULTI_TENANT` or `ENABLE_MULTI_TENANTS`
|
||||
2. Both tenant AND KB context must be set for document/graph API calls to work
|
||||
3. The WebUI uses localStorage to persist tenant/KB selection - must clear for fresh testing
|
||||
4. React strict mode causes double initialization - need refs to prevent duplicate API calls
|
||||
39
logs/2025-01-27-12-30-multitenant-filtering-api-fix.md
Normal file
39
logs/2025-01-27-12-30-multitenant-filtering-api-fix.md
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
# Task Log: Multi-Tenant Filtering & API Tab Fix
|
||||
|
||||
**Date:** 2025-01-27 12:30
|
||||
**Mode:** beastmode
|
||||
|
||||
## Todo List Status
|
||||
|
||||
- [x] Step 1: Pipeline/Document routes multi-tenant filtering (completed earlier)
|
||||
- [x] Step 2: Graph routes multi-tenant filtering (10 endpoints updated)
|
||||
- [x] Step 3: Query/Retrieval routes multi-tenant filtering (3 endpoints updated)
|
||||
- [x] Step 4: Update lightrag_server.py to pass rag_manager to all routes
|
||||
- [x] Step 5: Fix API tab visibility - Add /static proxy for Swagger UI assets
|
||||
- [ ] Step 6: Restart Vite dev server to apply proxy configuration change (user action required)
|
||||
|
||||
## Actions
|
||||
- Updated `graph_routes.py`: Added `get_tenant_rag` dependency to all 10 graph endpoints
|
||||
- Updated `query_routes.py`: Added `get_tenant_rag` dependency to 3 query endpoints (`/query`, `/query/stream`, `/query/data`)
|
||||
- Updated `lightrag_server.py`: Pass `rag_manager` to `create_graph_routes()` and `create_query_routes()`
|
||||
- Updated `vite.config.ts`: Added `/static` proxy to fix Swagger UI asset loading
|
||||
|
||||
## Decisions
|
||||
- Used same multi-tenant pattern across all routes: `get_tenant_rag` dependency returns tenant-specific LightRAG instance
|
||||
- API tab fix: Added `/static` proxy rather than changing Swagger UI configuration
|
||||
|
||||
## Next Steps
|
||||
- Restart Vite dev server (Ctrl+C and `bun run dev`) to apply proxy configuration change
|
||||
- Test API tab now shows Swagger UI
|
||||
- Test graph/retrieval operations filter by KB when switching knowledgebases
|
||||
|
||||
## Lessons/Insights
|
||||
- Swagger UI loads from `/docs` but assets come from `/static/swagger-ui/*` - both paths need proxying
|
||||
- Vite's `base: '/webui/'` setting redirects non-proxied paths, causing 404s for Swagger assets
|
||||
- Proxy configuration changes require dev server restart to take effect
|
||||
|
||||
## Files Modified
|
||||
1. `lightrag/api/routers/graph_routes.py` - Multi-tenant support for all graph endpoints
|
||||
2. `lightrag/api/routers/query_routes.py` - Multi-tenant support for all query endpoints
|
||||
3. `lightrag/api/lightrag_server.py` - Pass rag_manager to graph and query route creators
|
||||
4. `lightrag_webui/vite.config.ts` - Added `/static` proxy for Swagger UI assets
|
||||
80
logs/2025-12-04-21-30-multi-tenant-implementation.md
Normal file
80
logs/2025-12-04-21-30-multi-tenant-implementation.md
Normal file
|
|
@ -0,0 +1,80 @@
|
|||
# Multi-Tenant Implementation and Testing Log
|
||||
|
||||
**Date:** 2025-12-04
|
||||
**Mode:** beastmode
|
||||
|
||||
## Summary
|
||||
|
||||
Successfully implemented and tested full multi-tenant support for LightRAG WebUI. Both multi-tenant and single-tenant modes are now fully functional.
|
||||
|
||||
## Actions Performed
|
||||
|
||||
1. **Fixed Vite proxy configuration** (`lightrag_webui/vite.config.ts`)
|
||||
- Changed from conditional `import.meta.env` (doesn't work in config) to proper `loadEnv`
|
||||
- Added comprehensive proxy rules for all API endpoints: `/api/v1/`, `/api/`, `/documents/`, `/query/`, `/graph/`, `/retrieval/`, `/health/`, `/auth-status/`, `/docs`, `/redoc`, `/openapi.json`
|
||||
- Used `127.0.0.1` instead of `localhost` to avoid IPv6 resolution issues
|
||||
- Removed circular dependency by inlining `webuiPrefix` constant
|
||||
|
||||
2. **Tested Multi-Tenant Mode**
|
||||
- Verified `/api/v1/tenants` endpoint returns tenant list (acme-corp, techstart)
|
||||
- Verified `/api/v1/knowledge-bases` endpoint returns KBs per tenant
|
||||
- Login page shows tenant dropdown with all available tenants
|
||||
- Tenant selection loads associated knowledge bases
|
||||
- KB dropdown allows switching between KBs within tenant
|
||||
- Switch Tenant modal shows tenant cards with stats (KBs, Docs, GB)
|
||||
- Tenant switching properly resets context and loads new KBs
|
||||
- Document upload works with proper tenant/KB context
|
||||
- Query/Retrieval tab sends requests with tenant/KB headers
|
||||
- Knowledge Graph tab loads with graph controls
|
||||
|
||||
3. **Tested Single-Tenant Mode**
|
||||
- Set `LIGHTRAG_MULTI_TENANT=false` in `.env`
|
||||
- WebUI auto-redirects to dashboard without tenant selection
|
||||
- Default tenant/KB ("default") used automatically
|
||||
- All tabs functional (Documents, Knowledge Graph, Retrieval, API)
|
||||
- No tenant selector shown in header
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- Used `127.0.0.1` instead of `localhost` in Vite proxy to ensure consistent IPv4 connections
|
||||
- Inlined `webuiPrefix` constant in vite.config.ts to avoid circular import with path alias
|
||||
- Started Vite with `node ./node_modules/vite/bin/vite.js < /dev/null &` to prevent TTY suspension issues
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `lightrag_webui/vite.config.ts` - Fixed proxy configuration and removed problematic import
|
||||
|
||||
## Test Results
|
||||
|
||||
| Test | Status |
|
||||
|------|--------|
|
||||
| Vite proxy for /api/v1/ routes | ✅ Pass |
|
||||
| Tenant list API | ✅ Pass |
|
||||
| Knowledge bases API | ✅ Pass |
|
||||
| Login page tenant dropdown | ✅ Pass |
|
||||
| Tenant selection and KB loading | ✅ Pass |
|
||||
| Tenant switching | ✅ Pass |
|
||||
| KB switching within tenant | ✅ Pass |
|
||||
| Document upload with tenant context | ✅ Pass |
|
||||
| Query/Retrieval with tenant context | ✅ Pass |
|
||||
| Knowledge Graph tab | ✅ Pass |
|
||||
| Single-tenant mode auto-redirect | ✅ Pass |
|
||||
| Single-tenant mode default KB | ✅ Pass |
|
||||
|
||||
## Known Issues / Minor Items
|
||||
|
||||
1. SwaggerUI static files not found at `/static/swagger-ui/` - not critical, only affects `/docs` page
|
||||
2. Some "KB context required but missing" errors during tenant switching - timing issue, doesn't affect functionality
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Consider adding loading spinner during tenant/KB switch to avoid timing issues
|
||||
- Add SwaggerUI static files to Vite proxy or configure fallback
|
||||
|
||||
## Environment
|
||||
|
||||
- Branch: `premerge/integration-upstream`
|
||||
- API Port: 9621
|
||||
- WebUI Port: 5173
|
||||
- PostgreSQL: 15432
|
||||
- Redis: 16379
|
||||
Loading…
Add table
Reference in a new issue