59e89772de
Remove legacy storage implementations and deprecated examples: - Delete FAISS, JSON, Memgraph, Milvus, MongoDB, Nano Vector DB, Neo4j, NetworkX, Qdrant, Redis storage backends - Remove Kubernetes deployment manifests and installation scripts - Delete unofficial examples for deprecated backends and offline deployment docs Streamline core infrastructure: - Consolidate storage layer to PostgreSQL-only implementation - Add full-text search caching with FTS cache module - Implement metrics collection and monitoring pipeline - Add explain and metrics API routes Modernize frontend and tooling: - Switch web UI to Bun with bun.lock, remove npm and pnpm lockfiles - Update Dockerfile for PostgreSQL-only deployment - Add Makefile for common development tasks - Update environment and configuration examples Enhance evaluation and testing capabilities: - Add prompt optimization with DSPy and auto-tuning - Implement ground truth regeneration and variant testing - Add prompt debugging and response comparison utilities - Expand test coverage with new integration scenarios Simplify dependencies and configuration: - Remove offline-specific requirement files - Update pyproject.toml with streamlined dependencies - Add Python version pinning with .python-version - Create project guidelines in CLAUDE.md and AGENTS.md
34 lines
No EOL
908 B
JSON
34 lines
No EOL
908 B
JSON
{
|
|
"iteration": 1,
|
|
"timestamp": "2025-12-10T20:07:34.866066",
|
|
"best_score": 0.831272470422976,
|
|
"variants": [
|
|
{
|
|
"variant_name": "baseline",
|
|
"source": "baseline",
|
|
"faithfulness": 0.9871794871794872,
|
|
"relevance": 0.5915555273031531,
|
|
"ragas_score": 0.8289299032289535,
|
|
"total_latency_ms": 32942.73299979977,
|
|
"query_count": 3
|
|
},
|
|
{
|
|
"variant_name": "llm_mutation",
|
|
"source": "llm_mutation",
|
|
"faithfulness": 1.0,
|
|
"relevance": 0.38231813833458217,
|
|
"ragas_score": 0.7529272553338329,
|
|
"total_latency_ms": 33619.1771668382,
|
|
"query_count": 3
|
|
},
|
|
{
|
|
"variant_name": "aggressive_mutation",
|
|
"source": "llm_mutation",
|
|
"faithfulness": 1.0,
|
|
"relevance": 0.5781811760574401,
|
|
"ragas_score": 0.831272470422976,
|
|
"total_latency_ms": 11504.546957905404,
|
|
"query_count": 3
|
|
}
|
|
]
|
|
} |