gmakstutis/LightRAG
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
LightRAG/lightrag
History
anouarbm 1ad0bf82f9 feat: add RAGAS evaluation framework for RAG quality assessment 
This contribution adds a comprehensive evaluation system using the RAGAS
framework to assess LightRAG's retrieval and generation quality.

Features:
- RAGEvaluator class with four key metrics:
  * Faithfulness: Answer accuracy vs context
  * Answer Relevance: Query-response alignment
  * Context Recall: Retrieval completeness
  * Context Precision: Retrieved context quality
- HTTP API integration for live system testing
- JSON and CSV report generation
- Configurable test datasets
- Complete documentation with examples
- Sample test dataset included

Changes:
- Added lightrag/evaluation/eval_rag_quality.py (RAGAS evaluator implementation)
- Added lightrag/evaluation/README.md (comprehensive documentation)
- Added lightrag/evaluation/__init__.py (package initialization)
- Updated pyproject.toml with optional 'evaluation' dependencies
- Updated .gitignore to exclude evaluation results directory

Installation:
pip install lightrag-hku[evaluation]

Dependencies:
- ragas>=0.3.7
- datasets>=4.3.0
- httpx>=0.28.1
- pytest>=8.4.2
- pytest-asyncio>=1.2.0
2025-11-01 21:36:39 +01:00
..
api Add PDF decryption support for password-protected files 2025-11-01 15:01:17 +08:00
evaluation feat: add RAGAS evaluation framework for RAG quality assessment 2025-11-01 21:36:39 +01:00
kg Add Qdrant legacy collection migration with workspace support 2025-10-30 19:16:33 +08:00
llm Add offline deployment support with cache management and layered deps 2025-10-11 10:28:14 +08:00
tools Add offline deployment support with cache management and layered deps 2025-10-11 10:28:14 +08:00
__init__.py Bump core version to 1.4.9.8 and API version to 0250 2025-10-31 05:23:48 +08:00
base.py Fix RayAnything compatible problem 2025-10-22 20:15:29 +08:00
constants.py Refactor entity merging with unified attribute merge function 2025-10-27 00:04:17 +08:00
exceptions.py Add Qdrant legacy collection migration with workspace support 2025-10-30 19:16:33 +08:00
lightrag.py Fix edge cleanup when deleting entities to prevent orphaned relationships 2025-10-31 02:36:15 +08:00
namespace.py Add entity/relation chunk tracking with configurable source ID limits 2025-10-20 15:24:15 +08:00
operate.py Rename function and variables for clarity in context building 2025-11-01 12:15:24 +08:00
prompt.py Fix typo in 'equipment' in prompt.py 2025-10-22 11:13:22 +08:00
rerank.py fix: Resolve default rerank config problem when env var missing 2025-08-23 01:07:59 +08:00
types.py Add is_truncated to graph query for NetworkX graph db 2025-04-02 22:12:20 +08:00
utils.py Enhance entity/relation editing with chunk tracking synchronization 2025-10-26 14:34:56 +08:00
utils_graph.py Improve entity merge logging by removing redundant message and fixing typo 2025-10-31 17:16:59 +08:00