ragflow/api/db/services
hsparks-codes 237a66913b
Feat: RAG evaluation (#11674)
### What problem does this PR solve?

Feature: This PR implements a comprehensive RAG evaluation framework to
address issue #11656.

**Problem**: Developers using RAGFlow lack systematic ways to measure
RAG accuracy and quality. They cannot objectively answer:
1. Are RAG results truly accurate?
2. How should configurations be adjusted to improve quality?
3. How to maintain and improve RAG performance over time?

**Solution**: This PR adds a complete evaluation system with:
- **Dataset & test case management** - Create ground truth datasets with
questions and expected answers
- **Automated evaluation** - Run RAG pipeline on test cases and compute
metrics
- **Comprehensive metrics** - Precision, recall, F1 score, MRR, hit rate
for retrieval quality
- **Smart recommendations** - Analyze results and suggest specific
configuration improvements (e.g., "increase top_k", "enable reranking")
- **20+ REST API endpoints** - Full CRUD operations for datasets, test
cases, and evaluation runs

**Impact**: Enables developers to objectively measure RAG quality,
identify issues, and systematically improve their RAG systems through
data-driven configuration tuning.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-12-03 17:00:58 +08:00
..
__init__.py Refactor: fix typos (#10200) 2025-09-25 12:05:43 +08:00
api_service.py Add time utils (#10849) 2025-10-28 19:09:14 +08:00
canvas_service.py Feat: add or logic operations for meta data filters. (#11404) 2025-11-20 14:31:12 +08:00
common_service.py Fix: add auto_parse to kb detail. (#11153) 2025-11-11 12:22:43 +08:00
connector_service.py feat: improve metadata handling in connector service (#11421) 2025-11-26 19:55:48 +08:00
conversation_service.py Move some constants to common (#11004) 2025-11-05 08:01:39 +08:00
dialog_service.py Feat:support tts in agent (#11675) 2025-12-03 12:03:59 +08:00
document_service.py Refa: make RAGFlow more asynchronous 2 (#11689) 2025-12-03 14:19:53 +08:00
evaluation_service.py Feat: RAG evaluation (#11674) 2025-12-03 17:00:58 +08:00
file2document_service.py Move some constants to common (#11004) 2025-11-05 08:01:39 +08:00
file_service.py Refa: make RAGFlow more asynchronous (#11601) 2025-12-01 14:24:06 +08:00
knowledgebase_service.py Feat: Alter flask to Quart for async API serving. (#11275) 2025-11-18 17:05:16 +08:00
langfuse_service.py Add time utils (#10849) 2025-10-28 19:09:14 +08:00
llm_service.py Fix: incorrect async chat streamly output (#11679) 2025-12-03 11:15:45 +08:00
mcp_server_service.py Fix typos: retrievaler -> retriever (#10372) 2025-10-10 09:17:36 +08:00
pipeline_operation_log_service.py Feat: add data source to pipleline logs . (#11075) 2025-11-07 11:43:59 +08:00
search_service.py Move some constants to common (#11004) 2025-11-05 08:01:39 +08:00
task_service.py Move api.settings to common.settings (#11036) 2025-11-06 09:36:38 +08:00
tenant_llm_service.py Move api.settings to common.settings (#11036) 2025-11-06 09:36:38 +08:00
user_canvas_version.py Fix typos: retrievaler -> retriever (#10372) 2025-10-10 09:17:36 +08:00
user_service.py Move api.settings to common.settings (#11036) 2025-11-06 09:36:38 +08:00