This guide explains how to evaluate quality when considering hybrid architectures (e.g., GLiNER + LLM):
- 3-tier evaluation pyramid: entity → relation → end-to-end RAG
- Gold standard dataset creation (manual annotation + pseudo-labeling)
- Precision/Recall/F1 metrics for entities and relations
- Integration with existing RAGAS evaluation framework
- Real-world case study with decision thresholds
- Quality vs speed tradeoff matrix
Key thresholds:
- Entity F1 drop < 5%
- Relation F1 drop < 3%
- RAGAS score drop < 2%
Helps users make informed decisions about optimization strategies.
