Commit graph

23 commits

Author SHA1 Message Date
Raphaël MANSUY
7eda6d0a4e cherry-pick d36be1f4 2025-12-04 19:19:23 +08:00
Raphaël MANSUY
248c389fda cherry-pick a73314a4 2025-12-04 19:19:23 +08:00
Raphaël MANSUY
921e15fb87 cherry-pick 2823f92f 2025-12-04 19:19:23 +08:00
Raphaël MANSUY
6b0fe03dcf cherry-pick 06b91d00 2025-12-04 19:19:23 +08:00
Raphaël MANSUY
7aa4af900a cherry-pick d4b8a229 2025-12-04 19:19:23 +08:00
Raphaël MANSUY
aba1c65d7f cherry-pick c358f405 2025-12-04 19:19:23 +08:00
Raphaël MANSUY
d761d8e6be cherry-pick 41c26a36 2025-12-04 19:19:23 +08:00
Raphaël MANSUY
4af5f6c759 cherry-pick 36694eb9 2025-12-04 19:19:23 +08:00
Raphaël MANSUY
7ce259fbb4 cherry-pick 0b5e3f9d 2025-12-04 19:19:23 +08:00
Raphaël MANSUY
eef3178064 cherry-pick 026bca00 2025-12-04 19:19:23 +08:00
Raphaël MANSUY
4b18e6b3e4 cherry-pick 1ad0bf82 2025-12-04 19:19:22 +08:00
Raphaël MANSUY
9a11d689a2 cherry-pick 7abc6877 2025-12-04 19:19:01 +08:00
Raphaël MANSUY
2debc9288d cherry-pick 6d61f70b 2025-12-04 19:19:00 +08:00
Raphaël MANSUY
cf0899c063 cherry-pick 9d69e8d7 2025-12-04 19:19:00 +08:00
Raphaël MANSUY
af49572886 cherry-pick 363f3051 2025-12-04 19:19:00 +08:00
Raphaël MANSUY
56b8806256 cherry-pick 9c057060 2025-12-04 19:14:25 +08:00
anouarbm
349c1945db Optimize RAGAS evaluation with parallel execution and chunk content enrichment
Added efficient RAG evaluation system with optimized API calls and comprehensive benchmarking.

Key Features:
- Single API call per evaluation (2x faster than before)
- Parallel evaluation based on MAX_ASYNC environment variable
- Chunk content enrichment in /query endpoint responses
- Comprehensive benchmark statistics (moyennes)
- NaN-safe metric calculations

API Changes:
- Added include_chunk_content parameter to QueryRequest (backward compatible)
- /query endpoint enriches references with actual chunk content when requested
- No breaking changes - default behavior unchanged

Evaluation Improvements:
- Parallel execution using asyncio.Semaphore (respects MAX_ASYNC)
- Shared HTTP client with connection pooling
- Proper timeout handling (3min connect, 5min read)
- Debug output for context retrieval verification
- Benchmark statistics with averages, min/max scores

Results:
- Moyenne RAGAS Score: 0.9772
- Perfect Faithfulness: 1.0000
- Perfect Context Recall: 1.0000
- Perfect Context Precision: 1.0000
- Excellent Answer Relevance: 0.9087

(cherry picked from commit 0bbef9814e)
2025-12-04 19:11:20 +08:00
anouarbm
ccdd3c2786 fixed ruff format of csv path
(cherry picked from commit b12b693a81)
2025-12-04 19:11:08 +08:00
anouarbm
949bfc4228 fix: Apply ruff formatting and rename test_dataset to sample_dataset
**Lint Fixes (ruff)**:
- Sort imports alphabetically (I001)
- Add blank line after import traceback (E302)
- Add trailing comma to dict literals (COM812)
- Reformat writer.writerow for readability (E501)

**Rename test_dataset.json → sample_dataset.json**:
- Avoids .gitignore pattern conflict (test_* is ignored)
- More descriptive name - it's a sample/template, not actual test data
- Updated all references in eval_rag_quality.py and README.md

Resolves lint-and-format CI check failure.
Addresses reviewer feedback about test dataset naming.

(cherry picked from commit 5cdb4b0ef2)
2025-12-04 19:11:08 +08:00
yangdx
7896c42fba Restructure semaphore control to manage entire evaluation pipeline
• Move rag_semaphore to wrap full function
• Increase RAG concurrency to 2x eval limit
• Prevent memory buildup from slow evals
• Keep eval_semaphore for RAGAS control

(cherry picked from commit e5abe9dd3d)
2025-12-04 19:09:02 +08:00
yangdx
c459caed26 Implement two-stage pipeline for RAG evaluation with separate semaphores
• Split RAG gen and eval stages
• Add rag_semaphore for stage 1
• Add eval_semaphore for stage 2
• Improve concurrency control
• Update connection pool limits

(cherry picked from commit 83715a3ac1)
2025-12-04 19:09:02 +08:00
ben moussa anouar
dd425e5513 Update lightrag/evaluation/eval_rag_quality.py for launguage
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
(cherry picked from commit 98f0464a31)
2025-12-04 19:09:02 +08:00
yangdx
dec282694c Update .env loading and add API authentication to RAG evaluator
• Load .env from current directory
• Support LIGHTRAG_API_KEY auth header
• Override=False for env precedence
• Add Bearer token to API requests
• Enable per-instance .env configs

(cherry picked from commit 72db042667)
2025-12-04 19:04:25 +08:00