gmakstutis/LightRAG
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Cleans up documentation and deployment scripts for consistency

Browse source 
Removes trailing whitespace and fixes minor formatting issues in Kubernetes deployment docs, storage report, and Helm chart files.

Standardizes indentation and spacing in Docker Compose and deployment shell scripts to improve readability and maintainability.

These edits improve documentation clarity and make deployment scripts more robust without altering functionality.

Relates to MLO-469
This commit is contained in:
Taddeus 2025-11-03 14:23:16 +02:00
parent 81dff560f1
commit 4d9342c8e1
20 changed files with 92 additions and 92 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

24
blueprints/KUBERNETES_DEPLOYMENT.md
View file

@ -26,17 +26,17 @@ graph TD
E[Secrets & API Keys]
end
end
F[Helm Chart Management]
G[External Access]
H[OpenAI API]
F --> A
F --> B
A --> B
A --> H
C --> G
style A fill:#e1f5fe
style B fill:#f3e5f5
style F fill:#e8f5e8
@ -102,7 +102,7 @@ kubectl wait --namespace lightrag \
-l app.kubernetes.io/name=postgresql \
--timeout=120s
# Wait for LightRAG to be ready
# Wait for LightRAG to be ready
kubectl wait --namespace lightrag \
--for=condition=ready pod \
-l app.kubernetes.io/name=lightrag-minimal \
@ -130,7 +130,7 @@ kubectl port-forward --namespace lightrag svc/lightrag-minimal 9621:9621 &
# - Secure passwords
# - Resource limits based on your needs
# Substitute environment variables
# Substitute environment variables
envsubst < values-prod.yaml > values-prod-final.yaml
# Deploy with production configuration
@ -187,7 +187,7 @@ ingress:
enabled: false
```
#### Production (values-prod.yaml)
#### Production (values-prod.yaml)
```yaml
# Production resources
resources:
@ -262,7 +262,7 @@ curl http://localhost:9621/documents
# LightRAG logs
kubectl logs --namespace lightrag -l app.kubernetes.io/name=lightrag-minimal -f
# PostgreSQL logs
# PostgreSQL logs
kubectl logs --namespace lightrag -l app.kubernetes.io/name=postgresql -f
```
@ -302,7 +302,7 @@ python load_docs.py /path/to/docs --no-test
- Example: `[DC] getting-started/installation.md`
**URLs Mode:**
- Uses live website URLs in query response references
- Uses live website URLs in query response references
- Provides clickable links in responses
- Better user experience with direct access to source material
- Example: `[DC] https://docs.apolo.us/index/getting-started/installation`
@ -321,7 +321,7 @@ docs/guides/deployment.md → https://docs.example.com/guides/deployme
**Setup Instructions:**
1. **Analyze your docs site URLs** - Note the exact path structure
2. **Create matching directories** - Mirror the URL structure locally
2. **Create matching directories** - Mirror the URL structure locally
3. **Place files correctly** - Remove `.md` from URL paths to match filenames
4. **Test URLs** - Verify a few links work before loading documents
@ -329,7 +329,7 @@ This ensures generated URLs in query responses are valid and clickable.
### Loader Features
- **Simple dependencies**: Only requires `httpx`
- **Simple dependencies**: Only requires `httpx`
- **Automatic discovery**: Finds all `.md` files recursively
- **Basic metadata**: Adds title, path, and source information
- **Progress tracking**: Shows loading progress with success/failure counts
@ -455,7 +455,7 @@ kubectl port-forward --namespace lightrag svc/lightrag-minimal 9621:9621
## 🎯 Advantages of This Approach
✅ **Pure Helm** - No manual kubectl apply commands
✅ **Integrated PostgreSQL** - Bitnami chart handles all PostgreSQL complexity
✅ **Integrated PostgreSQL** - Bitnami chart handles all PostgreSQL complexity
✅ **pgvector Support** - Automatic extension creation via initdb scripts
✅ **Environment Flexibility** - Separate values files for dev/prod
✅ **Production Ready** - Built-in scaling, security, monitoring hooks
@ -503,4 +503,4 @@ This deployment process has been thoroughly validated through complete teardown
📊 Documentation: Ready for loading with included loader script
```
This approach provides a production-ready, maintainable solution for deploying LightRAG to any Kubernetes cluster with confidence in its reliability and repeatability.
This approach provides a production-ready, maintainable solution for deploying LightRAG to any Kubernetes cluster with confidence in its reliability and repeatability.

18
blueprints/REPORT.md
View file

@ -4,7 +4,7 @@
LightRAG supports a modular storage architecture with 4 distinct storage types that can be mixed and matched:
- **Graph Storage**: Knowledge graph relationships
- **Vector Storage**: Document embeddings
- **Vector Storage**: Document embeddings
- **KV Storage**: Key-value pairs and metadata
- **Document Status Storage**: Document processing status
@ -44,7 +44,7 @@ This report analyzes 25+ storage implementations across 8 database technologies
**Configuration:**
```yaml
LIGHTRAG_KV_STORAGE: PGKVStorage
LIGHTRAG_VECTOR_STORAGE: PGVectorStorage
LIGHTRAG_VECTOR_STORAGE: PGVectorStorage
LIGHTRAG_DOC_STATUS_STORAGE: PGDocStatusStorage
LIGHTRAG_GRAPH_STORAGE: PGGraphStorage # Requires AGE extension
```
@ -175,7 +175,7 @@ LIGHTRAG_DOC_STATUS_STORAGE: MongoDocStatusStorage
```yaml
LIGHTRAG_GRAPH_STORAGE: Neo4JStorage
LIGHTRAG_VECTOR_STORAGE: MilvusVectorDBStorage
LIGHTRAG_VECTOR_STORAGE: MilvusVectorDBStorage
LIGHTRAG_KV_STORAGE: RedisKVStorage
LIGHTRAG_DOC_STATUS_STORAGE: PGDocStatusStorage
```
@ -209,7 +209,7 @@ graph LR
```yaml
LIGHTRAG_GRAPH_STORAGE: NetworkXStorage
LIGHTRAG_VECTOR_STORAGE: QdrantVectorDBStorage
LIGHTRAG_KV_STORAGE: RedisKVStorage
LIGHTRAG_KV_STORAGE: RedisKVStorage
LIGHTRAG_DOC_STATUS_STORAGE: PGDocStatusStorage
```
@ -339,7 +339,7 @@ graph LR
### Vector Search Performance (Approximate)
| Implementation | Small (1K docs) | Medium (100K docs) | Large (1M+ docs) | Memory Usage |
|---------------|-----------------|--------------------|-----------------|--------------|
|---------------|-----------------|--------------------|-----------------|--------------|
| MilvusVectorDB | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | High |
| QdrantVectorDB | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | Medium |
| PGVectorStorage | ⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐ | Medium |
@ -354,7 +354,7 @@ graph LR
| PGGraphStorage | ⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐⭐ |
| NetworkXStorage | ⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐ | ⭐⭐ |
### KV Operations Performance
### KV Operations Performance
| Implementation | Read Speed | Write Speed | Concurrency | Persistence |
|---------------|------------|-------------|-------------|-------------|
| RedisKVStorage | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ |
@ -399,7 +399,7 @@ graph LR
### 📚 **Documentation/Knowledge Base**
- **Small (<10K docs)**: Development Stack
- **Medium (<100K docs)**: Minimal Stack
- **Medium (<100K docs)**: Minimal Stack
- **Large (>100K docs)**: Balanced Stack
### 🔬 **Research/Analytics**
@ -422,7 +422,7 @@ graph LR
The **Minimal Stack** (PostgreSQL + NetworkX) provides the best balance of performance, complexity, and cost for most use cases. It offers:
- ✅ Production-ready reliability
- ✅ Reasonable performance for medium-scale deployments
- ✅ Reasonable performance for medium-scale deployments
- ✅ Low operational overhead
- ✅ Clear upgrade path to specialized components
@ -435,4 +435,4 @@ The modular architecture allows gradual optimization based on actual performance
---
*Report generated based on LightRAG v1.3.7 implementation analysis*
*Report generated based on LightRAG v1.3.7 implementation analysis*

28
deploy-stacks.sh
View file

@ -54,31 +54,31 @@ create_directories() {
deploy_stack() {
local stack=$1
local compose_file="docker-compose.${stack}.yml"
if [ ! -f "$compose_file" ]; then
print_error "Compose file $compose_file not found!"
return 1
fi
print_status "Deploying $stack stack..."
# Stop any existing containers
docker-compose -f "$compose_file" down 2>/dev/null || true
# Start the stack
docker-compose -f "$compose_file" up -d
if [ $? -eq 0 ]; then
print_success "$stack stack deployed successfully!"
# Show running services
echo ""
print_status "Running services:"
docker-compose -f "$compose_file" ps
# Wait a bit for services to start
sleep 5
# Check LightRAG health
print_status "Checking LightRAG health..."
for i in {1..30}; do
@ -93,7 +93,7 @@ deploy_stack() {
echo -n "."
sleep 2
done
print_warning "LightRAG health check timed out. Check logs with:"
echo "docker-compose -f $compose_file logs lightrag"
else
@ -106,12 +106,12 @@ deploy_stack() {
stop_stack() {
local stack=$1
local compose_file="docker-compose.${stack}.yml"
if [ ! -f "$compose_file" ]; then
print_error "Compose file $compose_file not found!"
return 1
fi
print_status "Stopping $stack stack..."
docker-compose -f "$compose_file" down
print_success "$stack stack stopped"
@ -121,12 +121,12 @@ stop_stack() {
cleanup_stack() {
local stack=$1
local compose_file="docker-compose.${stack}.yml"
if [ ! -f "$compose_file" ]; then
print_error "Compose file $compose_file not found!"
return 1
fi
print_warning "This will remove all data for $stack stack. Are you sure? (y/N)"
read -r response
if [[ "$response" =~ ^([yY][eE][sS]|[yY])$ ]]; then
@ -214,4 +214,4 @@ main() {
}
# Run main function with all arguments
main "$@"
main "$@"

12
docker-compose.all-in-one.yml
View file

@ -17,19 +17,19 @@ services:
LIGHTRAG_VECTOR_STORAGE: QdrantVectorDBStorage
LIGHTRAG_KV_STORAGE: RedisKVStorage
LIGHTRAG_DOC_STATUS_STORAGE: MongoDocStatusStorage
# Service Connection Details
NEO4J_URI: bolt://neo4j:7687
NEO4J_USERNAME: neo4j
NEO4J_PASSWORD: lightrag_neo4j_pass
QDRANT_URL: http://qdrant:6333
REDIS_URI: redis://redis:6379
MONGO_URI: mongodb://lightrag_user:lightrag_pass@mongodb:27017/lightrag?authSource=admin
MONGO_DATABASE: lightrag
# Performance settings
MAX_ASYNC: 6
MAX_TOKENS: 32768
@ -145,4 +145,4 @@ volumes:
networks:
lightrag-aio-network:
driver: bridge
driver: bridge

12
docker-compose.balanced.yml
View file

@ -1,7 +1,7 @@
version: '3.8'
# Production Balanced Stack
# NetworkX + Qdrant + Redis + PostgreSQL
# NetworkX + Qdrant + Redis + PostgreSQL
# Best for: Production deployments prioritizing simplicity
services:
@ -17,18 +17,18 @@ services:
LIGHTRAG_VECTOR_STORAGE: QdrantVectorDBStorage
LIGHTRAG_KV_STORAGE: RedisKVStorage
LIGHTRAG_DOC_STATUS_STORAGE: PGDocStatusStorage
# Service Connection Details
QDRANT_URL: http://qdrant:6333
REDIS_URI: redis://redis:6379
POSTGRES_HOST: postgres
POSTGRES_PORT: 5432
POSTGRES_USER: lightrag_user
POSTGRES_PASSWORD: lightrag_pass
POSTGRES_DATABASE: lightrag
# Performance settings
MAX_ASYNC: 6
MAX_TOKENS: 32768
@ -119,4 +119,4 @@ volumes:
networks:
lightrag-balanced-network:
driver: bridge
driver: bridge

6
docker-compose.development.yml
View file

@ -17,14 +17,14 @@ services:
LIGHTRAG_VECTOR_STORAGE: NanoVectorDBStorage
LIGHTRAG_KV_STORAGE: JsonKVStorage
LIGHTRAG_DOC_STATUS_STORAGE: JsonDocStatusStorage
# Development settings
MAX_ASYNC: 2
MAX_TOKENS: 16384
ENABLE_LLM_CACHE: true
ENABLE_LLM_CACHE_FOR_EXTRACT: false
LOG_LEVEL: DEBUG
# No external database connections needed
ports:
- "9621:9621"
@ -78,4 +78,4 @@ volumes:
networks:
lightrag-dev-network:
driver: bridge
driver: bridge

12
docker-compose.high-performance.yml
View file

@ -17,23 +17,23 @@ services:
LIGHTRAG_VECTOR_STORAGE: MilvusVectorDBStorage
LIGHTRAG_KV_STORAGE: RedisKVStorage
LIGHTRAG_DOC_STATUS_STORAGE: PGDocStatusStorage
# Service Connection Details
NEO4J_URI: bolt://neo4j:7687
NEO4J_USERNAME: neo4j
NEO4J_PASSWORD: lightrag_neo4j_pass
MILVUS_URI: http://milvus-standalone:19530
MILVUS_DB_NAME: lightrag
REDIS_URI: redis://redis:6379
POSTGRES_HOST: postgres
POSTGRES_PORT: 5432
POSTGRES_USER: lightrag_user
POSTGRES_PASSWORD: lightrag_pass
POSTGRES_DATABASE: lightrag
# Performance optimizations
MAX_ASYNC: 8
MAX_TOKENS: 32768
@ -196,4 +196,4 @@ volumes:
networks:
lightrag-hp-network:
driver: bridge
driver: bridge

2
docker-compose.minimal.yml
View file

@ -53,4 +53,4 @@ volumes:
networks:
default:
name: lightrag-minimal-network
name: lightrag-minimal-network

2
init-db/01-init.sql
View file

@ -1,2 +1,2 @@
-- Enable the vector extension for pgvector
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS vector;

2
k8s-deploy/lightrag-minimal/Chart.yaml
View file

@ -10,4 +10,4 @@ dependencies:
- name: postgresql
version: "~12.0.0"
repository: "https://charts.bitnami.com/bitnami"
condition: postgresql.enabled
condition: postgresql.enabled

24
k8s-deploy/lightrag-minimal/README.md
View file

@ -66,11 +66,11 @@ env:
EMBEDDING_BINDING_HOST: "https://api.openai.com/v1"
EMBEDDING_MODEL: "text-embedding-ada-002"
EMBEDDING_DIM: "1536"
# Conservative concurrency (prevents API errors)
MAX_ASYNC: "4"
MAX_PARALLEL_INSERT: "2"
# LLM Configuration
ENABLE_LLM_CACHE: "true"
ENABLE_LLM_CACHE_FOR_EXTRACT: "true"
@ -207,7 +207,7 @@ python ../../../load_docs.py ../docs --endpoint http://localhost:9621
python ../../../load_docs.py ../apolo-copilot/docs/official-apolo-documentation/docs \
--mode urls --base-url https://docs.apolo.us/index/ --endpoint http://localhost:9621
# Load custom documentation with URL references
# Load custom documentation with URL references
python ../../../load_docs.py /path/to/docs \
--mode urls --base-url https://your-docs.example.com/docs/ --endpoint http://localhost:9621
```
@ -241,7 +241,7 @@ docs/
**URL Mapping Rules:**
- `.md` extension is removed from URLs
- `README.md` files map to their directory URL
- `README.md` files map to their directory URL
- Subdirectories become URL path segments
- File and folder names should match URL slugs exactly
@ -284,7 +284,7 @@ Both modes produce similar output with different reference formats:
```
**URLs Mode References:**
```
```
### References
- [DC] https://docs.apolo.us/index/getting-started/installation
- [KG] https://docs.apolo.us/index/administration/cluster-setup
@ -332,13 +332,13 @@ kubectl delete namespace lightrag
## Validated Features
✅ **Pure Helm Deployment** - No manual kubectl apply commands needed
✅ **PostgreSQL with pgvector** - Automatic extension creation via proper image
✅ **Environment Flexibility** - Separate dev/prod configurations
✅ **Document Loading** - Working API with `file_source` parameter
✅ **Conservative Concurrency** - Prevents OpenAI API rate limiting
✅ **Health Monitoring** - Comprehensive health checks and status endpoints
✅ **Persistent Storage** - Data survives pod restarts and cluster updates
✅ **Pure Helm Deployment** - No manual kubectl apply commands needed
✅ **PostgreSQL with pgvector** - Automatic extension creation via proper image
✅ **Environment Flexibility** - Separate dev/prod configurations
✅ **Document Loading** - Working API with `file_source` parameter
✅ **Conservative Concurrency** - Prevents OpenAI API rate limiting
✅ **Health Monitoring** - Comprehensive health checks and status endpoints
✅ **Persistent Storage** - Data survives pod restarts and cluster updates
## Comparison with Docker Compose

10
k8s-deploy/lightrag-minimal/templates/NOTES.txt
View file

@ -15,7 +15,7 @@
{{- else }}
# Start port forwarding (run in background):
kubectl port-forward --namespace {{ .Release.Namespace }} svc/{{ include "lightrag-minimal.fullname" . }} 9621:9621 &
🌐 Web UI: http://localhost:9621/webui
📚 API Docs: http://localhost:9621/docs
🔍 Health Check: http://localhost:9621/health
@ -27,10 +27,10 @@
4. Load your documents:
# Install httpx if needed: pip install httpx
# Files mode (file path references) - Default
python ../../../load_docs.py /path/to/your/docs --endpoint http://localhost:9621
# URLs mode (website URL references) - Recommended for public docs
# Note: Local file structure must match your docs site URL structure
python ../../../load_docs.py /path/to/docs --mode urls \
@ -39,7 +39,7 @@
5. Monitor your deployment:
# Check pods status
kubectl get pods --namespace {{ .Release.Namespace }}
# View logs
kubectl logs --namespace {{ .Release.Namespace }} -l app.kubernetes.io/name={{ include "lightrag-minimal.name" . }} -f
@ -51,4 +51,4 @@ Configuration Summary:
🤖 LLM Model: {{ .Values.env.LLM_MODEL }}
📊 Concurrency: MAX_ASYNC={{ .Values.env.MAX_ASYNC }}, MAX_PARALLEL_INSERT={{ .Values.env.MAX_PARALLEL_INSERT }}
📖 For detailed usage instructions, see the README.md in the chart directory.
📖 For detailed usage instructions, see the README.md in the chart directory.

18
k8s-deploy/lightrag-minimal/templates/deployment.yaml
View file

@ -43,13 +43,13 @@ spec:
value: {{ .Values.env.HOST | quote }}
- name: PORT
value: {{ .Values.env.PORT | quote }}
# Web UI configuration
- name: WEBUI_TITLE
value: {{ .Values.env.WEBUI_TITLE | quote }}
- name: WEBUI_DESCRIPTION
value: {{ .Values.env.WEBUI_DESCRIPTION | quote }}
# LLM configuration
- name: LLM_BINDING
value: {{ .Values.env.LLM_BINDING | quote }}
@ -62,7 +62,7 @@ spec:
secretKeyRef:
name: {{ include "lightrag-minimal.secretName" . }}
key: llm-api-key
# Embedding configuration
- name: EMBEDDING_BINDING
value: {{ .Values.env.EMBEDDING_BINDING | quote }}
@ -106,7 +106,7 @@ spec:
value: {{ .Values.env.LIGHTRAG_DOC_STATUS_STORAGE | quote }}
- name: LIGHTRAG_GRAPH_STORAGE
value: {{ .Values.env.LIGHTRAG_GRAPH_STORAGE | quote }}
# PostgreSQL configuration
- name: POSTGRES_HOST
value: {{ include "lightrag-minimal.postgresqlHost" . | quote }}
@ -123,7 +123,7 @@ spec:
value: {{ .Values.env.POSTGRES_DATABASE | quote }}
- name: POSTGRES_WORKSPACE
value: {{ .Values.env.POSTGRES_WORKSPACE | quote }}
{{- if .Values.healthCheck.enabled }}
livenessProbe:
httpGet:
@ -142,10 +142,10 @@ spec:
timeoutSeconds: 5
failureThreshold: 3
{{- end }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- if .Values.persistence.enabled }}
volumeMounts:
- name: rag-storage
@ -153,7 +153,7 @@ spec:
- name: inputs
mountPath: /app/inputs
{{- end }}
{{- if .Values.persistence.enabled }}
volumes:
- name: rag-storage
@ -163,7 +163,7 @@ spec:
persistentVolumeClaim:
claimName: {{ include "lightrag-minimal.fullname" . }}-inputs
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}

2
k8s-deploy/lightrag-minimal/templates/hpa.yaml
View file

@ -29,4 +29,4 @@ spec:
type: Utilization
averageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}
{{- end }}
{{- end }}
{{- end }}

2
k8s-deploy/lightrag-minimal/templates/ingress.yaml
View file

@ -41,4 +41,4 @@ spec:
port:
name: http
{{- end }}
{{- end }}
{{- end }}

2
k8s-deploy/lightrag-minimal/templates/pvc.yaml
View file

@ -32,4 +32,4 @@ spec:
{{- if .Values.persistence.inputs.storageClass }}
storageClassName: {{ .Values.persistence.inputs.storageClass }}
{{- end }}
{{- end }}
{{- end }}

2
k8s-deploy/lightrag-minimal/templates/service.yaml
View file

@ -12,4 +12,4 @@ spec:
protocol: TCP
name: http
selector:
{{- include "lightrag-minimal.selectorLabels" . | nindent 4 }}
{{- include "lightrag-minimal.selectorLabels" . | nindent 4 }}

2
k8s-deploy/lightrag-minimal/templates/serviceaccount.yaml
View file

@ -9,4 +9,4 @@ metadata:
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}
{{- end }}

2
k8s-deploy/lightrag/templates/ingress.yaml
View file

@ -51,4 +51,4 @@ spec:
secretName: {{ .secretName }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}

2
k8s-deploy/lightrag/values.yaml
View file

@ -81,4 +81,4 @@ tolerations:
affinity: {}
nodeSelector: {}
nodeSelector: {}