Learning path · Evaluation & Quality · 76
Faithfulness and Relevance
Faithfulness measures whether answers are supported by context; relevance measures whether they address the question.
Why it matters
- Orthogonal failures need different fixes—prompt vs retrieval.
- Core RAG metrics alongside Precision@K and Recall@K.
- Drive abstention policies when either score is low.
Key ideas
- Supported claims
- Question alignment
- Abstention thresholds
Top resources
- 01DocsExploding Gradients
RAGAS
Why this resource. Definitions of faithfulness vs relevancy on retrieved context.
Covers in this concept
- faithfulness
- answer relevancy
- context precision
- 02PaperLiu et al.
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
Why this resource. Rubric scoring when a single metric is not enough.
Covers in this concept
- criteria
An answer can be faithful but irrelevant (quotes the right doc but misses the question) or relevant but unfaithful (addresses the ask using invented facts). Score both separately with judges or NLI models. Set production thresholds to trigger shorter replies, extra retrieval, or human escalation. Report slice metrics in dashboards executives actually read. Tune abstention copy for low scores—users respond better to scoped "here is what we know" answers than empty errors.
Updated 2026-08-09 · Full learning path