LightRAG/e2e/README.md

E2E Multi-Tenant Isolation Tests

This directory contains end-to-end tests to verify the isolation of data and processing pipelines in the multi-tenant LightRAG implementation.

Quick Start

# Run all tests with file backend (default)
./e2e/run_tests.sh

# Interactive mode - guided test selection
./e2e/run_tests.sh -i

# Show help
./e2e/run_tests.sh --help

Test Runner Features

The enhanced test runner (run_tests.sh) provides:

• 🎮 Interactive Mode: Menu-driven selection of backends and tests
• 💾 Multiple Backends: Support for file, postgres, or all backends
• 🧪 Test Selection: Run specific tests or all tests
• 🔍 Dry Run: Preview configuration without executing
• 📊 Verbose/Quiet Modes: Control output verbosity
• 🎨 Colored Output: Easy-to-read results with emojis

Available Tests

Test	File	Description
isolation	test_multitenant_isolation.py	Tests data isolation between tenants
deletion	test_deletion.py	Tests document deletion and cleanup
mixed	test_mixed_operations.py	Tests interleaved tenant operations

Available Backends

Backend	Description
file	File-based storage (JSON, NetworkX, NanoVectorDB) - Default
postgres	PostgreSQL with pgvector for production storage
all	Run tests on both backends

Usage Examples

Basic Usage

# Run all tests with file backend (default)
./e2e/run_tests.sh

# Run all tests with PostgreSQL backend
./e2e/run_tests.sh -b postgres

# Run tests on all backends
./e2e/run_tests.sh -b all

# Run with OpenAI models (requires OPENAI_API_KEY)
./e2e/run_tests.sh --openai

# Reset database/storage before running tests
./e2e/run_tests.sh --reset-db

Test Selection

# Run only isolation test
./e2e/run_tests.sh -t isolation

# Run isolation and deletion tests
./e2e/run_tests.sh -t isolation,deletion

# Run specific test with postgres backend
./e2e/run_tests.sh -b postgres -t isolation

Interactive Mode

# Launch interactive menu
./e2e/run_tests.sh -i

The interactive mode guides you through:

1. Backend selection
2. Test selection
3. Advanced options (verbose, keep server)
4. LLM model configuration

Preview Configuration

# Dry run to see configuration without executing
./e2e/run_tests.sh --dry-run -b postgres -t isolation,deletion

Custom Model Configuration

# Use OpenAI models (gpt-4o-mini + text-embedding-3-small)
./e2e/run_tests.sh --openai

# Use custom LLM model (Ollama)
./e2e/run_tests.sh -m llama3.1:8b

# Use custom embedding model (Ollama)
./e2e/run_tests.sh -e nomic-embed-text:latest -d 768

# Combined: OpenAI with reset and postgres backend
./e2e/run_tests.sh --openai --reset-db -b postgres

Server Management

# Keep server running after tests (for debugging)
./e2e/run_tests.sh --keep-server

# Skip server management (use existing server)
./e2e/run_tests.sh --skip-server

# Use custom port
./e2e/run_tests.sh -p 8080

Output Control

# Verbose output with debug info
./e2e/run_tests.sh -v

# Quiet mode (minimal output)
./e2e/run_tests.sh -q

# Show live server logs during test execution
./e2e/run_tests.sh --logs

# Full example with logs, OpenAI, reset, and postgres
./e2e/run_tests.sh --openai --reset-db -b postgres --logs

OpenAI Configuration

# Use OpenAI models (requires OPENAI_API_KEY environment variable)
export OPENAI_API_KEY="sk-..."
./e2e/run_tests.sh --openai

# The --openai flag sets:
# - LLM: gpt-5-nano (OpenAI's fast model)
# - Embedding: text-embedding-3-small (1536 dimensions)

Database Reset

# Reset storage before running tests (clean slate)
./e2e/run_tests.sh --reset-db

# For file backend: clears rag_storage/ directory
# For postgres: Docker volume is removed and recreated

# Combined with OpenAI for a fresh run
./e2e/run_tests.sh --openai --reset-db -b postgres

Command Line Options

Option	Description	Default
-b, --backend	Storage backend (file, postgres, all)	file
-t, --tests	Comma-separated test list	all
-m, --llm-model	LLM model name	gpt-oss:20b
--llm-binding	LLM binding type	ollama
-e, --embedding-model	Embedding model	bge-m3:latest
--embedding-binding	Embedding binding type	ollama
-d, --dim	Embedding dimension	1024
-p, --port	Server port	9621
-i, --interactive	Interactive mode	-
-v, --verbose	Verbose output	-
-q, --quiet	Quiet mode	-
--dry-run	Preview without executing	-
--skip-server	Don't manage server	-
--keep-server	Keep server running	-
-l, --list	List tests and backends	-
--openai	Use OpenAI models (gpt-5-nano + text-embedding-3-small)	-
--reset-db	Reset database/storage before tests	-
--skip-docker	Don't manage Docker containers	-
--keep-docker	Keep Docker containers running	-
--logs	Show live server logs during tests	-
-h, --help	Show help	-
--version	Show version	-

Environment Variables

Variable	Description	Default
OPENAI_API_KEY	OpenAI API key (required for --openai)	-
LIGHTRAG_API_URL	API URL	http://localhost:9621
AUTH_USER	Admin username	admin
AUTH_PASS	Admin password	admin123
POSTGRES_HOST	PostgreSQL host	localhost
POSTGRES_PORT	PostgreSQL port	5432
POSTGRES_USER	PostgreSQL user	lightrag
POSTGRES_PASSWORD	PostgreSQL password	-
POSTGRES_DATABASE	PostgreSQL database	lightrag_multitenant

Test Details

Multi-Tenant Isolation Test (isolation)

Tests data isolation between tenants:

1. Creates two distinct tenants (Tenant A and Tenant B)
2. Creates a Knowledge Base (KB) for each tenant
3. Ingests a unique "secret" document into each tenant's KB
4. Waits for indexing to complete
5. Verifies that Tenant A can retrieve its secret but not Tenant B's secret
6. Verifies that Tenant B can retrieve its secret but not Tenant A's secret

Document Deletion Test (deletion)

Tests document deletion functionality:

1. Creates a tenant and knowledge base
2. Ingests two documents
3. Verifies both documents are queryable
4. Deletes one document
5. Verifies deleted document is no longer retrievable
6. Verifies the other document still exists

Mixed Operations Test (mixed)

Tests interleaved operations across tenants:

1. Creates two tenants with separate KBs
2. Performs interleaved ingestion operations
3. Verifies cross-tenant isolation during concurrent operations
4. Tests deletion in one tenant doesn't affect the other
5. Verifies data integrity throughout

Prerequisites

• Python 3.10+
• requests library installed (pip install requests)
• LightRAG installed in the environment
• Ollama running locally (or configured via environment variables)
• For PostgreSQL tests: PostgreSQL with pgvector extension

Exit Codes

Code	Meaning
0	All tests passed
1	One or more tests failed
2	Configuration or setup error

Troubleshooting

Server Issues

• Check server.log in the project root if the server fails to start
• Use --verbose flag to see detailed server startup logs
• Try --keep-server to inspect server state after tests

Timeout Issues

• If indexing takes too long, check if LLM/Embedding service is responsive
• Increase timeout by modifying SERVER_TIMEOUT in the script

PostgreSQL Issues

• Ensure PostgreSQL is running with pgvector extension
• Verify connection settings in environment variables
• Check that the database exists and user has proper permissions

Legacy Script

The original run_isolation_test.sh is still available for backward compatibility but run_tests.sh is recommended for new usage.