da9070ecf7
Remove deprecated storage backends and Kubernetes deployment configuration: - Delete unused storage implementations: FAISS, JSON, Memgraph, Milvus, MongoDB, Nano Vector DB, Neo4j, NetworkX, Qdrant, Redis - Remove Kubernetes deployment manifests and installation scripts - Delete legacy examples for deprecated backends - Consolidate to PostgreSQL-only storage backend Streamline dependencies and add new capabilities: - Remove deprecated code documentation and migration guides - Add full-text search caching layer with FTS cache module - Implement metrics collection and monitoring pipeline - Add explain and metrics API routes - Simplify configuration with PostgreSQL-focused setup Update documentation and configuration: - Rewrite README to focus on supported features - Update environment and configuration examples - Remove Kubernetes-specific documentation - Add new utility scripts for PDF uploads and pipeline monitoring
34 lines
3.4 KiB
JSON
34 lines
3.4 KiB
JSON
{
|
|
"test_cases": [
|
|
{
|
|
"question": "How does LightRAG solve the hallucination problem in large language models?",
|
|
"ground_truth": "LightRAG solves the hallucination problem by combining large language models with external knowledge retrieval. The framework ensures accurate responses by grounding LLM outputs in actual documents. LightRAG provides contextual responses that reduce hallucinations significantly.",
|
|
"project": "lightrag_evaluation_sample"
|
|
},
|
|
{
|
|
"question": "What are the three main components required in a RAG system?",
|
|
"ground_truth": "A RAG system requires three main components: a retrieval system (vector database or search engine) to find relevant documents, an embedding model to convert text into vector representations for similarity search, and a large language model (LLM) to generate responses based on retrieved context.",
|
|
"project": "lightrag_evaluation_sample"
|
|
},
|
|
{
|
|
"question": "How does LightRAG's retrieval performance compare to traditional RAG approaches?",
|
|
"ground_truth": "LightRAG delivers faster retrieval performance than traditional RAG approaches. The framework optimizes document retrieval operations for speed. Traditional RAG systems often suffer from slow query response times. LightRAG achieves high quality results with improved performance. The framework combines speed with accuracy in retrieval operations, prioritizing ease of use without sacrificing quality.",
|
|
"project": "lightrag_evaluation_sample"
|
|
},
|
|
{
|
|
"question": "What vector databases does LightRAG support and what are their key characteristics?",
|
|
"ground_truth": "LightRAG now standardizes on PostgreSQL with pgvector for vectors and AGE-compatible graph storage, simplifying setup and focusing tuning and operations on a single, reliable backend.",
|
|
"project": "lightrag_evaluation_sample"
|
|
},
|
|
{
|
|
"question": "What are the four key metrics for evaluating RAG system quality and what does each metric measure?",
|
|
"ground_truth": "RAG system quality is measured through four key metrics: Faithfulness measures whether answers are factually grounded in retrieved context and detects hallucinations. Answer Relevance measures how well answers address the user question and evaluates response appropriateness. Context Recall measures completeness of retrieval and whether all relevant information was retrieved from documents. Context Precision measures quality and relevance of retrieved documents without noise or irrelevant content.",
|
|
"project": "lightrag_evaluation_sample"
|
|
},
|
|
{
|
|
"question": "What are the core benefits of LightRAG and how does it improve upon traditional RAG systems?",
|
|
"ground_truth": "LightRAG offers five core benefits: accuracy through document-grounded responses, up-to-date information without model retraining, domain expertise through specialized document collections, cost-effectiveness by avoiding expensive fine-tuning, and transparency by showing source documents. Compared to traditional RAG systems, LightRAG provides a simpler API with intuitive interfaces, faster retrieval performance with optimized operations, better integration with multiple vector database backends for flexible selection, and optimized prompting strategies with refined templates. LightRAG prioritizes ease of use while maintaining quality and combines speed with accuracy.",
|
|
"project": "lightrag_evaluation_sample"
|
|
}
|
|
]
|
|
}
|