LightRAG/docs/archives/adr/006-architecture-diagrams-alternatives.md

ADR 006: Architecture Diagrams and Alternatives Analysis

Status: Proposed

Proposed Architecture Diagram

┌─────────────────────────────────────────────────────────────────────────────┐
│                         LightRAG Multi-Tenant System                        │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                               │
│  ┌──────────────────────────────────────────────────────────────────┐      │
│  │                      FastAPI Application                         │      │
│  ├──────────────────────────────────────────────────────────────────┤      │
│  │                                                                   │      │
│  │  ┌─────────────────────────────────────────────────────────┐    │      │
│  │  │         Request Middleware Layer                        │    │      │
│  │  ├─────────────────────────────────────────────────────────┤    │      │
│  │  │ • CORS Middleware                                      │    │      │
│  │  │ • HTTPS Redirect                                       │    │      │
│  │  │ • Rate Limiting (per tenant)                           │    │      │
│  │  │ • Request Logging & Audit                              │    │      │
│  │  │ • Idempotency Key Handling                             │    │      │
│  │  └─────────────────────────────────────────────────────────┘    │      │
│  │                          ↓                                        │      │
│  │  ┌─────────────────────────────────────────────────────────┐    │      │
│  │  │      Authentication & Tenant Context Extraction        │    │      │
│  │  ├─────────────────────────────────────────────────────────┤    │      │
│  │  │ 1. Parse JWT token or API key from headers             │    │      │
│  │  │ 2. Validate signature and expiration                   │    │      │
│  │  │ 3. Extract tenant_id, kb_id, user_id, permissions      │    │      │
│  │  │ 4. Verify token.tenant_id == path.tenant_id            │    │      │
│  │  │ 5. Verify user can access kb_id                        │    │      │
│  │  │ → Returns TenantContext object                          │    │      │
│  │  └─────────────────────────────────────────────────────────┘    │      │
│  │                          ↓                                        │      │
│  │  ┌─────────────────────────────────────────────────────────┐    │      │
│  │  │         API Routing Layer                               │    │      │
│  │  ├─────────────────────────────────────────────────────────┤    │      │
│  │  │ /api/v1/tenants/{tenant_id}/                           │    │      │
│  │  │ ├─ knowledge-bases/{kb_id}/documents/*                │    │      │
│  │  │ ├─ knowledge-bases/{kb_id}/query*                     │    │      │
│  │  │ ├─ knowledge-bases/{kb_id}/graph/*                    │    │      │
│  │  │ ├─ knowledge-bases/*                                  │    │      │
│  │  │ └─ api-keys/*                                         │    │      │
│  │  └─────────────────────────────────────────────────────────┘    │      │
│  │                          ↓                                        │      │
│  │  ┌─────────────────────────────────────────────────────────┐    │      │
│  │  │    Request Handlers (with TenantContext injected)       │    │      │
│  │  ├─────────────────────────────────────────────────────────┤    │      │
│  │  │ • Validate permissions on TenantContext                │    │      │
│  │  │ • Get tenant-specific RAG instance                     │    │      │
│  │  │ • Pass context to business logic                       │    │      │
│  │  │ • Return response with audit trail                     │    │      │
│  │  └─────────────────────────────────────────────────────────┘    │      │
│  │                                                                   │      │
│  └──────────────────────────────────────────────────────────────────┘      │
│                                                                               │
│  ┌──────────────────────────────────────────────────────────────────┐      │
│  │              Tenant-Aware LightRAG Instance Manager              │      │
│  ├──────────────────────────────────────────────────────────────────┤      │
│  │                                                                   │      │
│  │  Instance Cache:                                                 │      │
│  │  ┌─────────────────────────────────────────────────────────┐    │      │
│  │  │ (tenant_1, kb_1) → LightRAG@memory                     │    │      │
│  │  │ (tenant_1, kb_2) → LightRAG@memory                     │    │      │
│  │  │ (tenant_2, kb_1) → LightRAG@memory                     │    │      │
│  │  │ (tenant_3, kb_1) → LightRAG@memory                     │    │      │
│  │  │ ...                                                     │    │      │
│  │  │ Max: 100 instances (configurable)                      │    │      │
│  │  └─────────────────────────────────────────────────────────┘    │      │
│  │                                                                   │      │
│  │  Each LightRAG instance:                                         │      │
│  │  • Uses tenant-specific configuration (LLM, embedding models)   │      │
│  │  • Works with dedicated namespace: {tenant_id}_{kb_id}          │      │
│  │  • Isolated storage connections                                 │      │
│  │  └─────────────────────────────────────────────────────────────┘    │      │
│  │                                                                   │      │
│  └──────────────────────────────────────────────────────────────────┘      │
│                                                                               │
│  ┌──────────────────────────────────────────────────────────────────┐      │
│  │              Storage Access Layer (with Tenant Filtering)        │      │
│  ├──────────────────────────────────────────────────────────────────┤      │
│  │                                                                   │      │
│  │  Query Modification:                                             │      │
│  │  Before:  SELECT * FROM documents WHERE doc_id = 'abc'          │      │
│  │  After:   SELECT * FROM documents                               │      │
│  │           WHERE tenant_id = 'acme' AND kb_id = 'docs'           │      │
│  │           AND doc_id = 'abc'                                    │      │
│  │                                                                   │      │
│  │  • All queries automatically scoped to current tenant/KB         │      │
│  │  • Prevents accidental cross-tenant data access                 │      │
│  │  • Storage layer enforces isolation (defense in depth)          │      │
│  │                                                                   │      │
│  └──────────────────────────────────────────────────────────────────┘      │
│                                                                               │
│  ┌──────────────────────────────────────────────────────────────────┐      │
│  │                    Storage Backends (Shared)                     │      │
│  ├──────────────────────────────────────────────────────────────────┤      │
│  │                                                                   │      │
│  │  ┌─────────────────┐  ┌─────────────┐  ┌────────────────────┐  │      │
│  │  │   PostgreSQL    │  │   Neo4j     │  │  Milvus/Qdrant    │  │      │
│  │  │  (Shared DB)    │  │  (Shared)   │  │   (Vector Store)   │  │      │
│  │  ├─────────────────┤  ├─────────────┤  ├────────────────────┤  │      │
│  │  │ • Documents     │  │ • Entities  │  │ • Embeddings       │  │      │
│  │  │ • Chunks        │  │ • Relations │  │ • Entity vectors   │  │      │
│  │  │ • Entities      │  │             │  │                    │  │      │
│  │  │ • API Keys      │  │ Each node   │  │ Each vector        │  │      │
│  │  │ • Tenants       │  │ tagged with │  │ tagged with        │  │      │
│  │  │ • KBs           │  │ tenant_id + │  │ tenant_id + kb_id  │  │      │
│  │  │                 │  │ kb_id       │  │                    │  │      │
│  │  │ Filtered by:    │  │             │  │ Filtered by:       │  │      │
│  │  │ tenant_id,      │  │ Filtered by:│  │ tenant_id,         │  │      │
│  │  │ kb_id in WHERE  │  │ tenant_id + │  │ kb_id in query     │  │      │
│  │  │                 │  │ kb_id       │  │                    │  │      │
│  │  └─────────────────┘  └─────────────┘  └────────────────────┘  │      │
│  │                                                                   │      │
│  │  All with tenant/KB isolation at schema/data level              │      │
│  └──────────────────────────────────────────────────────────────────┘      │
│                                                                               │
└─────────────────────────────────────────────────────────────────────────────┘

Data Flow Diagrams

Query Execution Flow

1. Client Request
   ├─ POST /api/v1/tenants/acme/knowledge-bases/docs/query
   ├─ Body: {"query": "What is..."}
   └─ Header: Authorization: Bearer <token>
          │
          ▼
2. Middleware Validation
   ├─ Extract tenant_id, kb_id from URL path
   ├─ Extract token from Authorization header
   ├─ Validate token signature and expiration
   ├─ Extract user_id, tenant_id_in_token, permissions
   └─ VERIFY: tenant_id (path) == tenant_id_in_token
          │
          ▼
3. Dependency Injection
   ├─ Create TenantContext(
   │   tenant_id="acme",
   │   kb_id="docs",
   │   user_id="john",
   │   role="editor",
   │   permissions={"query:run": true}
   └─ )
          │
          ▼
4. Handler Authorization
   ├─ Check TenantContext.permissions["query:run"] == true
   └─ If false → 403 Forbidden
          │
          ▼
5. Get RAG Instance
   ├─ RAGManager.get_instance(tenant_id="acme", kb_id="docs")
   ├─ Check cache → Found → Use cached instance
   └─ (If not cached: create new with tenant config)
          │
          ▼
6. Execute Query
   ├─ RAG.aquery(query="What is...", tenant_context=context)
   │  └─ All internal queries will include tenant/kb filters:
   │     └─ Storage layer automatically adds:
   │        WHERE tenant_id='acme' AND kb_id='docs'
          │
          ▼
7. Storage Layer Filtering
   ├─ Vector search: Find embeddings WHERE tenant_id='acme' AND kb_id='docs'
   ├─ Graph query: Match entities {tenant_id:'acme', kb_id:'docs'}
   ├─ KV lookup: Get items with key prefix 'acme:docs:'
   └─ Returns only acme/docs data (NO cross-tenant leakage possible)
          │
          ▼
8. Response Generation
   ├─ RAG generates response from filtered data
   ├─ Response object created
   └─ Handler receives response with TenantContext
          │
          ▼
9. Audit Logging
   ├─ Log: {
   │   user_id: "john",
   │   tenant_id: "acme",
   │   kb_id: "docs",
   │   action: "query_executed",
   │   status: "success",
   │   timestamp: <now>
   └─ }
          │
          ▼
10. Response Returned to Client
    └─ HTTP 200 with query result

Document Upload Flow

1. Client uploads document
   ├─ POST /api/v1/tenants/acme/knowledge-bases/docs/documents/add
   ├─ File: document.pdf
   └─ Header: Authorization: Bearer <token>
          │
          ▼
2. Authentication & Authorization
   ├─ Validate token, extract TenantContext
   ├─ Check permission: document:create
   └─ Verify tenant_id matches path and token
          │
          ▼
3. File Validation
   ├─ Check file type (PDF, DOCX, etc.)
   ├─ Check file size < quota
   ├─ Sanitize filename
   └─ Generate unique doc_id
          │
          ▼
4. Queue Document Processing
   ├─ Store temp file: /{working_dir}/{tenant_id}/{kb_id}/__tmp__/{doc_id}
   ├─ Create DocStatus record with status="processing"
   ├─ Return to client: {status: "processing", track_id: "..."}
   └─ Start async processing task
          │
          ▼
5. Async Document Processing (background task)
   ├─ Get RAG instance for (acme, docs)
   ├─ Insert document:
   │  └─ RAG.ainsert(file_path, tenant_id="acme", kb_id="docs")
   │     └─ Internal processing automatically tags data with:
   │        └─ tenant_id="acme", kb_id="docs"
   │
   ├─ Update DocStatus:
   │  ├─ status → "success"
   │  ├─ chunks_processed → 42
   │  └─ entities_extracted → 15
   │
   └─ Move file: __tmp__ → {kb_id}/documents/
          │
          ▼
6. Storage Writes (tenant-scoped)
   ├─ PostgreSQL:
   │  └─ INSERT INTO chunks (tenant_id, kb_id, doc_id, content)
   │     VALUES ('acme', 'docs', 'doc-123', '...')
   │
   ├─ Neo4j:
   │  └─ CREATE (e:Entity {tenant_id:'acme', kb_id:'docs', name:'...'})-[:IN_KB]->(kb)
   │
   └─ Milvus:
      └─ Insert vector with metadata: {tenant_id:'acme', kb_id:'docs'}
          │
          ▼
7. Client Polls for Status
   ├─ GET /api/v1/tenants/acme/knowledge-bases/docs/documents/{doc_id}/status
   ├─ Returns: {status: "success", chunks: 42, entities: 15}
   └─ Client confirms upload complete

Alternatives Considered

Alternative 1: Separate Database Per Tenant

Architecture:

• Each tenant gets dedicated PostgreSQL database
• Separate Neo4j instances per tenant
• Separate Milvus collections per tenant

Tenant A Server → PostgreSQL A
                → Neo4j A
                → Milvus A

Tenant B Server → PostgreSQL B
                → Neo4j B
                → Milvus B

Pros:

• Maximum isolation (physical separation)
• Easier compliance (HIPAA, GDPR)
• Better disaster recovery per tenant
• Easier scaling (scale out per tenant)

Cons:

• ❌ Massive operational overhead
  • Each database needs separate backup, upgrade, monitoring
  • 100 tenants = 100 databases to manage
  • Database licensing costs multiply (100x more expensive)
• ❌ Complex deployment & maintenance
  • Infrastructure-as-Code becomes complex
  • Database credentials management nightmare
  • Harder debugging with distributed databases
• ❌ Impossible resource sharing
  • Cannot leverage shared compute resources
  • Cannot optimize resource usage globally
  • Waste of resources (each DB has minimum overhead)
• ❌ Cross-tenant features impossible
  • Data sharing between tenants difficult
  • Consolidated reporting/analytics hard to implement

Decision: REJECTED Too expensive and operationally complex for moderate scale.

---

Alternative 2: Dedicated Server Per Tenant

Architecture:

• Each tenant runs own LightRAG instance
• Own Python process, own configurations
• Own memory/CPU allocation

Tenant A    → LightRAG Process A (port 9621)
Tenant B    → LightRAG Process B (port 9622)
Tenant C    → LightRAG Process C (port 9623)

Pros:

• Complete isolation (separate processes)
• Easy to manage per-tenant configs
• Can use different models per tenant

Cons:

• ❌ Massive resource waste
  • Minimum ~500MB RAM per instance × 100 tenants = 50GB+ RAM
  • Minimum CPU overhead per process
• ❌ Extremely expensive at scale
  • 100 tenants × 4GB allocated = 400GB RAM needed
  • Infrastructure costs prohibitive
• ❌ Operational nightmare
  • 100 processes to monitor
  • 100 upgrades/patches to manage
  • Complex deployment orchestration
• ❌ Poor utilization
  • Most tenants underutilize their resources
  • Cannot rebalance resources dynamically
  • Peak loads unpredictable per tenant

Decision: REJECTED Not economically viable for enterprise deployments.

---

Alternative 3: Simple Workspace Rename (No Knowledge Base)

Architecture:

• Rename "workspace" to "tenant"
• No KB concept
• Assume 1 KB per tenant

POST /api/v1/workspaces/{workspace_id}/query
→ becomes
POST /api/v1/tenants/{tenant_id}/query

Pros:

• Minimal code changes
• Backward compatible
• Quick implementation (1 week)

Cons:

• ❌ No knowledge base isolation
  • Tenant with multiple unrelated KBs must share config
  • Cannot have tenant-specific KB settings
  • All data mixed together
• ❌ Cannot enforce cross-tenant access prevention
  • Workspace is just a directory/field
  • No API-level enforcement
  • Easy to make mistakes
• ❌ No RBAC
  • Cannot grant access to specific KBs
  • All-or-nothing tenant access
  • No fine-grained permissions
• ❌ No tenant-specific configuration
  • All tenants must use same LLM/embedding models
  • Cannot customize per tenant needs
• ❌ Limited compliance features
  • No audit trails of who accessed what
  • Difficult to enforce data residency
  • No resource quotas

Decision: REJECTED Doesn't meet business requirements for true multi-tenancy.

---

Alternative 4: Shared Single LightRAG for All Tenants

Architecture:

• One LightRAG instance for all tenants
• Single namespace, single graph
• Tenant filtering only at API layer

API Layer → Filters query by tenant → Single LightRAG Instance

Pros:

• Minimal resource usage
• Single deployment
• Simple to maintain

Cons:

• ❌ Data isolation risk is CRITICAL
  • Single point of failure for all tenants
  • One query mistake → cross-tenant data leak
  • Cannot be patched without affecting all
• ❌ Performance bottleneck
  • Single instance cannot scale with tenants
  • All LLM calls compete for resources
  • All embedding calls serialized
• ❌ Tenant-specific configuration impossible
  • All tenants share same models
  • Cannot customize chunk size, top_k, etc per tenant
• ❌ No blast radius isolation
  • One tenant's bad data can corrupt all
  • One tenant's quota exhaustion affects all
• ❌ Compliance impossible
  • Data residency requirements: cannot guarantee where data is
  • GDPR right to deletion: must delete entire system
  • Audit requirements: cannot track per-tenant operations

Decision: REJECTED Unacceptable security and operational risks.

---

Alternative 5: Sharding by Tenant Hash

Architecture:

• Hash tenant ID
• Route to specific shard server
• Multiple instances with different tenant ranges

Tenant Hash % 3
├─ Shard 0: LightRAG A (tenants 0, 3, 6, 9...)
├─ Shard 1: LightRAG B (tenants 1, 4, 7, 10...)
└─ Shard 2: LightRAG C (tenants 2, 5, 8, 11...)

Pros:

• Distributes load across instances
• Better than single instance
• Can grow to 3+ instances

Cons:

• ❌ Breaks operational simplicity
  • Need load balancer + routing logic
  • Shards must be preconfigured
  • Adding tenant requires determining shard
• ❌ Rebalancing is complex
  • Adding new shard requires data migration
  • Tenant addition might change shard assignment
  • Hotspots impossible to fix dynamically
• ❌ Doesn't reduce fundamental costs
  • Still need multiple instances
  • Each instance has full overhead
  • Only slightly better than per-tenant instances
• ❌ More complex than multi-tenant single instance
  • Routing logic adds latency
  • Debugging harder (data could be on any shard)
  • Cross-shard features harder to implement

Decision: REJECTED Introduces complexity without enough benefit over single instance per tenant approach.

---

Comparison Table

Approach	Isolation	Cost	Complexity	Scalability	Selected
Proposed: Single Instance Multi-Tenant	✓ Good	✓ Low	✓ Medium	✓ Excellent	✓ YES
Alt 1: DB Per Tenant	✓✓ Perfect	✗✗ 100x	✗✗ Very High	✗ Limited	✗
Alt 2: Server Per Tenant	✓ Good	✗✗ 50x	✗ High	✗ Limited	✗
Alt 3: Workspace Rename	~ Weak	✓ Very Low	✓ Very Low	✓ Good	✗
Alt 4: Single Instance	✗ Poor	✓ Very Low	✓ Very Low	✗ Poor	✗
Alt 5: Sharding	✓ Good	✗ 10-20x	✗✗ High	✓ Good	✗

Why This Approach Wins

The proposed single instance, multi-tenant, multi-KB architecture offers the optimal balance:

1. Security: Complete tenant isolation through multiple layers
2. Cost: Efficient resource sharing (100 tenants ≈ 1.1x cost of single tenant)
3. Complexity: Manageable (dependency injection handles most complexity)
4. Scalability: Single instance can serve 100s of tenants, scales vertically well
5. Compliance: Audit trails and data isolation support compliance needs
6. Features: Supports RBAC, per-tenant config, resource quotas

---

Document Version: 1.0
Last Updated: 2025-11-20
Related Files: 001-multi-tenant-architecture-overview.md