1ad0bf82f9
This contribution adds a comprehensive evaluation system using the RAGAS framework to assess LightRAG's retrieval and generation quality. Features: - RAGEvaluator class with four key metrics: * Faithfulness: Answer accuracy vs context * Answer Relevance: Query-response alignment * Context Recall: Retrieval completeness * Context Precision: Retrieved context quality - HTTP API integration for live system testing - JSON and CSV report generation - Configurable test datasets - Complete documentation with examples - Sample test dataset included Changes: - Added lightrag/evaluation/eval_rag_quality.py (RAGAS evaluator implementation) - Added lightrag/evaluation/README.md (comprehensive documentation) - Added lightrag/evaluation/__init__.py (package initialization) - Updated pyproject.toml with optional 'evaluation' dependencies - Updated .gitignore to exclude evaluation results directory Installation: pip install lightrag-hku[evaluation] Dependencies: - ragas>=0.3.7 - datasets>=4.3.0 - httpx>=0.28.1 - pytest>=8.4.2 - pytest-asyncio>=1.2.0
74 lines
826 B
Text
74 lines
826 B
Text
# Python-related files
|
|
__pycache__/
|
|
*.py[cod]
|
|
*.egg-info/
|
|
.eggs/
|
|
*.tgz
|
|
*.tar.gz
|
|
*.ini
|
|
|
|
# Virtual Environment
|
|
.venv/
|
|
venv/
|
|
|
|
# Enviroment Variable Files
|
|
.env
|
|
|
|
# Build / Distribution
|
|
dist/
|
|
build/
|
|
site/
|
|
|
|
# Logs / Reports
|
|
*.log
|
|
*.log.*
|
|
*.logfire
|
|
*.coverage/
|
|
log/
|
|
|
|
# Caches
|
|
.cache/
|
|
.mypy_cache/
|
|
.pytest_cache/
|
|
.ruff_cache/
|
|
.gradio/
|
|
.history/
|
|
temp/
|
|
|
|
# IDE / Editor Files
|
|
.idea/
|
|
.vscode/
|
|
.vscode/settings.json
|
|
|
|
# Framework-specific files
|
|
local_neo4jWorkDir/
|
|
neo4jWorkDir/
|
|
|
|
# Data & Storage
|
|
inputs/
|
|
output/
|
|
rag_storage/
|
|
data/
|
|
|
|
# Evaluation results
|
|
lightrag/evaluation/results/
|
|
|
|
# Miscellaneous
|
|
.DS_Store
|
|
TODO.md
|
|
ignore_this.txt
|
|
*.ignore.*
|
|
|
|
# Project-specific files
|
|
/dickens*/
|
|
/book.txt
|
|
download_models_hf.py
|
|
|
|
# Frontend build output (built during PyPI release)
|
|
/lightrag/api/webui/
|
|
|
|
# unit-test files
|
|
test_*
|
|
|
|
# Cline files
|
|
memory-bank
|