2b292d4924
* Remove outdated documentation files: Quick Start Guide, Apache AGE Analysis, and Scratchpad. * Add multi-tenant testing strategy and ADR index documentation - Introduced ADR 008 detailing the multi-tenant testing strategy for the ./starter environment, covering compatibility and multi-tenant modes, testing scenarios, and implementation details. - Created a comprehensive ADR index (README.md) summarizing all architecture decision records related to the multi-tenant implementation, including purpose, key sections, and reading paths for different roles. * feat(docs): Add comprehensive multi-tenancy guide and README for LightRAG Enterprise - Introduced `0008-multi-tenancy.md` detailing multi-tenancy architecture, key concepts, roles, permissions, configuration, and API endpoints. - Created `README.md` as the main documentation index, outlining features, quick start, system overview, and deployment options. - Documented the LightRAG architecture, storage backends, LLM integrations, and query modes. - Established a task log (`2025-01-21-lightrag-documentation-log.md`) summarizing documentation creation actions, decisions, and insights.
337 lines
12 KiB
Markdown
337 lines
12 KiB
Markdown
# Multi-Tenant Implementation Audit - Final Report
|
|
|
|
**Date:** November 29, 2025
|
|
**Auditor:** Automated Audit Process
|
|
**Version:** 1.0
|
|
|
|
---
|
|
|
|
## Executive Summary
|
|
|
|
This audit reviewed the multi-tenant implementation of the LightRAG stack from Web UI through REST API to Storage layer. The implementation uses a **workspace-based isolation model** where `workspace = tenant_id/kb_id`.
|
|
|
|
### Overall Assessment: ⚠️ FUNCTIONAL WITH CONCERNS
|
|
|
|
The multi-tenant implementation is **functionally correct** but has several security and architectural concerns that should be addressed before production deployment.
|
|
|
|
---
|
|
|
|
## Architecture Overview
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Web UI (React) │
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
|
|
│ │ TenantStore │ │ KBSelector │ │ Axios Interceptor │ │
|
|
│ │ (Zustand) │ │ │ │ X-Tenant-ID Header │ │
|
|
│ └──────────────┘ └──────────────┘ └──────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
HTTP Headers (X-Tenant-ID, X-KB-ID)
|
|
│
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ REST API (FastAPI) │
|
|
│ ┌──────────────────┐ ┌────────────────┐ ┌─────────────────┐ │
|
|
│ │ get_tenant_ │ │ TenantContext │ │ TenantRAG │ │
|
|
│ │ context() │──│ (dataclass) │──│ Manager │ │
|
|
│ │ dependency │ │ │ │ (per-tenant RAG)│ │
|
|
│ └──────────────────┘ └────────────────┘ └─────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
│
|
|
workspace = tenant_id/kb_id
|
|
│
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Storage Layer │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ PostgreSQL Tables with workspace column │ │
|
|
│ │ PRIMARY KEY (workspace, id) │ │
|
|
│ │ │ │
|
|
│ │ Tables: LIGHTRAG_DOC_FULL, LIGHTRAG_DOC_CHUNKS, │ │
|
|
│ │ LIGHTRAG_VDB_*, LIGHTRAG_TENANTS, etc. │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ ┌─────────────────┐ ┌─────────────────┐ ┌────────────────┐ │
|
|
│ │ Redis │ │ Apache AGE │ │ Vector Store │ │
|
|
│ │ (key prefix) │ │ (graph per ws) │ │ (ws filter) │ │
|
|
│ └─────────────────┘ └─────────────────┘ └────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Test Results Summary
|
|
|
|
### Unit Tests: ✅ ALL PASSED
|
|
|
|
| Test Suite | Tests | Result |
|
|
|------------|-------|--------|
|
|
| test_multitenant_e2e.py | 32 | ✅ PASSED |
|
|
| test_tenant_security.py | 11 | ✅ PASSED |
|
|
| test_multi_tenant_backends.py | 37 | ✅ PASSED |
|
|
| test_tenant_storage_phase3.py | 22 | ✅ PASSED |
|
|
| **TOTAL** | **102** | **✅ ALL PASSED** |
|
|
|
|
### Key Verifications
|
|
- ✅ Cross-tenant data access prevention
|
|
- ✅ Knowledge base isolation within tenants
|
|
- ✅ Role-based permission enforcement
|
|
- ✅ Composite key uniqueness
|
|
- ✅ Redis namespace isolation
|
|
- ✅ Vector ID uniqueness per tenant
|
|
|
|
---
|
|
|
|
## Security Findings
|
|
|
|
### Critical Findings
|
|
|
|
| ID | Severity | Component | Finding |
|
|
|----|----------|-----------|---------|
|
|
| SEC-001 | 🔴 HIGH | API | `get_tenant_context_optional()` allows global RAG fallback |
|
|
| SEC-002 | 🔴 HIGH | API | Admin user bypass in `tenant_service.py` |
|
|
| SEC-003 | 🟡 MEDIUM | API | `user_id` parameter optional in RAG manager |
|
|
| SEC-004 | 🟡 MEDIUM | Storage | No PostgreSQL Row-Level Security (RLS) |
|
|
| SEC-005 | 🟢 LOW | Storage | Tenant context set for logging only |
|
|
| SEC-006 | 🟢 LOW | WebUI | localStorage used for token storage |
|
|
|
|
### SEC-001: Global RAG Fallback (HIGH)
|
|
|
|
**Location:** `lightrag/api/routers/document_routes.py`, `query_routes.py`
|
|
|
|
**Issue:** Using `get_tenant_context_optional` allows requests without tenant headers to use global RAG:
|
|
|
|
```python
|
|
async def get_tenant_rag(
|
|
tenant_context: Optional[TenantContext] = Depends(get_tenant_context_optional)
|
|
) -> LightRAG:
|
|
if rag_manager and tenant_context:
|
|
return await rag_manager.get_rag_instance(...)
|
|
return rag # Falls back to global RAG!
|
|
```
|
|
|
|
**Risk:** Data leakage if global RAG contains multi-tenant data.
|
|
|
|
**Recommendation:** Use `get_tenant_context` (required) for all data endpoints in multi-tenant mode.
|
|
|
|
### SEC-002: Admin User Bypass (HIGH)
|
|
|
|
**Location:** `lightrag/services/tenant_service.py`
|
|
|
|
**Issue:**
|
|
```python
|
|
if user_id.lower() == "admin":
|
|
return True # Bypass all access checks
|
|
```
|
|
|
|
**Risk:** Any user with username "admin" can access any tenant's data.
|
|
|
|
**Recommendation:** Implement proper super-admin role with separate authentication.
|
|
|
|
### SEC-003: Optional User ID (MEDIUM)
|
|
|
|
**Location:** `lightrag/tenant_rag_manager.py`
|
|
|
|
**Issue:** `user_id` parameter is optional, allowing access without authentication check.
|
|
|
|
**Recommendation:** Deprecate unauthenticated access path.
|
|
|
|
### SEC-004: No Row-Level Security (MEDIUM)
|
|
|
|
**Location:** PostgreSQL Tables
|
|
|
|
**Issue:** Tenant isolation relies entirely on application-level `workspace` filtering. No database-level enforcement.
|
|
|
|
**Recommendation:** Enable PostgreSQL RLS as defense-in-depth:
|
|
```sql
|
|
ALTER TABLE LIGHTRAG_DOC_FULL ENABLE ROW LEVEL SECURITY;
|
|
CREATE POLICY tenant_isolation ON LIGHTRAG_DOC_FULL
|
|
USING (workspace = current_setting('app.current_workspace'));
|
|
```
|
|
|
|
---
|
|
|
|
## Architectural Findings
|
|
|
|
### ARCH-001: Workspace Encoding
|
|
|
|
**Pattern:** `workspace = "{tenant_id}/{kb_id}"`
|
|
|
|
**Strengths:**
|
|
- Simple, hierarchical namespace
|
|
- Clear data ownership
|
|
- Compatible with file-based storage
|
|
|
|
**Concerns:**
|
|
- No explicit validation of workspace format
|
|
- Potential collision if tenant IDs contain `/`
|
|
|
|
### ARCH-002: Context Propagation
|
|
|
|
**Pattern:** Python `ContextVar` for tenant context
|
|
|
|
```python
|
|
tenant_id_var: ContextVar[Optional[str]] = ContextVar("tenant_id")
|
|
```
|
|
|
|
**Current Use:** Set for PostgreSQL session logging, but NOT used for security.
|
|
|
|
**Recommendation:** Use context variable for RLS enforcement.
|
|
|
|
### ARCH-003: RAG Instance Caching
|
|
|
|
**Pattern:** LRU cache keyed by `(tenant_id, kb_id)`
|
|
|
|
**Strengths:**
|
|
- Memory efficient
|
|
- Fast subsequent access
|
|
|
|
**Concerns:**
|
|
- Cache size limit may cause thrashing with many tenants
|
|
- No cache invalidation on tenant deletion
|
|
|
|
---
|
|
|
|
## Web UI Findings
|
|
|
|
### UI-001: Tenant State Management
|
|
|
|
**Location:** `lightrag_webui/src/stores/tenant.ts`
|
|
|
|
**Status:** ✅ Well implemented
|
|
|
|
- Zustand store with persistence
|
|
- Automatic header injection via Axios interceptor
|
|
- Console logging for debugging
|
|
|
|
### UI-002: Document State Clearing
|
|
|
|
**Location:** `lightrag_webui/src/features/DocumentManager.tsx`
|
|
|
|
**Recent Fix:** State clearing on tenant/KB change
|
|
|
|
```typescript
|
|
useEffect(() => {
|
|
setCurrentPageDocs([]);
|
|
setDocs(null);
|
|
setStatusCounts({ all: 0 });
|
|
// ...
|
|
}, [selectedTenant?.tenant_id, selectedKB?.kb_id]);
|
|
```
|
|
|
|
**Status:** ✅ Fixed (per task log dated 2025-02-25)
|
|
|
|
### UI-003: Token Storage
|
|
|
|
**Finding:** Authentication token stored in localStorage
|
|
|
|
**Risk:** XSS vulnerability could expose tokens
|
|
|
|
**Recommendation:** Use httpOnly cookies for production
|
|
|
|
---
|
|
|
|
## Recommendations
|
|
|
|
### Priority 1: Critical Security Fixes
|
|
|
|
1. **Replace `get_tenant_context_optional` with `get_tenant_context`** for all data endpoints
|
|
2. **Remove admin bypass** in `tenant_service.py`
|
|
3. **Make `user_id` required** in RAG manager
|
|
|
|
### Priority 2: Defense-in-Depth
|
|
|
|
4. **Enable PostgreSQL RLS** for all tenant-scoped tables
|
|
5. **Validate workspace format** to prevent injection
|
|
6. **Add audit logging** for cross-tenant access attempts
|
|
|
|
### Priority 3: Operational Improvements
|
|
|
|
7. **Add cache invalidation** on tenant/KB deletion
|
|
8. **Implement rate limiting** per tenant
|
|
9. **Add metrics/monitoring** for tenant isolation verification
|
|
|
|
---
|
|
|
|
## Test Protocol for Manual Verification
|
|
|
|
### Prerequisites
|
|
```bash
|
|
# Start test databases
|
|
docker compose -f docker-compose.test-db.yml up -d
|
|
|
|
# Verify containers
|
|
docker ps --filter "name=lightrag-audit"
|
|
```
|
|
|
|
### Test Scenarios
|
|
|
|
#### Scenario 1: Cross-Tenant Data Access Prevention
|
|
1. Create Tenant A and KB
|
|
2. Upload document to Tenant A
|
|
3. Attempt to list documents with Tenant B headers
|
|
4. **Expected:** Empty list for Tenant B
|
|
|
|
#### Scenario 2: Missing Tenant Header
|
|
1. Send document list request without X-Tenant-ID header
|
|
2. **Expected:** 400 Bad Request OR empty response (not global data)
|
|
|
|
#### Scenario 3: Query Isolation
|
|
1. Insert data into Tenant A KB
|
|
2. Query with Tenant B context
|
|
3. **Expected:** Query returns no results from Tenant A
|
|
|
|
### Test Script
|
|
```bash
|
|
# Run API isolation tests
|
|
python docs/action_plan/multitenant-audit/scripts/test_api_isolation.py
|
|
```
|
|
|
|
---
|
|
|
|
## Conclusion
|
|
|
|
The LightRAG multi-tenant implementation provides **functional tenant isolation** through:
|
|
- Workspace-based table partitioning
|
|
- Per-tenant RAG instance management
|
|
- Header-based tenant context extraction
|
|
- Role-based permission model
|
|
|
|
However, the implementation has **security gaps** that must be addressed:
|
|
1. Optional tenant context fallback to global RAG
|
|
2. Admin user bypass in access control
|
|
3. No database-level security (RLS)
|
|
|
|
### Recommended Next Steps
|
|
|
|
1. 🔴 **Immediate:** Fix SEC-001 and SEC-002 before production
|
|
2. 🟡 **Short-term:** Enable PostgreSQL RLS
|
|
3. 🟢 **Long-term:** Add comprehensive audit logging and monitoring
|
|
|
|
---
|
|
|
|
## Appendices
|
|
|
|
### A. Files Reviewed
|
|
|
|
| Layer | Files |
|
|
|-------|-------|
|
|
| Web UI | `tenant.ts`, `client.ts`, `DocumentManager.tsx` |
|
|
| API | `dependencies.py`, `document_routes.py`, `query_routes.py`, `tenant_routes.py` |
|
|
| Service | `tenant_service.py`, `tenant_rag_manager.py` |
|
|
| Storage | `postgres_impl.py`, `utils_context.py`, `namespace.py` |
|
|
| Tests | `test_multitenant_e2e.py`, `test_tenant_security.py`, `test_multi_tenant_backends.py` |
|
|
|
|
### B. Test Environment
|
|
|
|
| Component | Version/Config |
|
|
|-----------|----------------|
|
|
| Python | 3.12.12 |
|
|
| PostgreSQL | pgvector/pgvector:pg16 |
|
|
| Redis | 7-alpine |
|
|
| OS | macOS (arm64) |
|
|
|
|
### C. Related Documentation
|
|
|
|
- `docs/0001-multi-tenant-architecture.md`
|
|
- `docs/0002-multi-tenant-visual-reference.md`
|
|
- `docs/0003-multi-tenant-documentation-index.md`
|
|
- `MULTITENANT_IMPLEMENTATION_SUMMARY.md`
|