Week 6 · RAG and evals · about 3 minutes
What you will be able to do
Block a Helix release when answers cite claims the retrieved chunks do not support.
The idea
Faithfulness asks: is every claim supported by the retrieved context? Fluency is not enough for clinical copilots.
Run an eval set before promote. If faithfulness drops below the agreed floor, the release does not ship.
def may_release(faithfulness: float, floor: float = 0.9) -> bool:
return faithfulness >= floorA green demo with a red faithfulness score is not a release.
Key takeaways
- Measure faithfulness on a gold set.
- Agree a floor in the MVV.
- Red score blocks the release.
Part of the Forward Deployed Engineer roadmap. New to the role? Start with what a Forward Deployed Engineer does.