Week 5 · LLM gateways · about 3 minutes
What you will be able to do
Cap spend per Helix tenant, then downgrade the model instead of failing the request.
The idea
Track spend per tenant. When a request would cross the ceiling, do not hard-fail the clinician — route to a cheaper model or a retrieval-only answer.
Ceilings protect the contract. Downgrades protect the user experience when the ceiling is hit.
def route(spend_usd: float, ceiling: float, estimate: float) -> str:
if spend_usd + estimate >= ceiling:
return "small"
return "primary"A shared uncapped key is a outage waiting for one busy tenant.
Key takeaways
- Meter spend per tenant.
- Hit the ceiling → cheaper path, not silence.
- Never share one uncapped key across tenants.
Part of the Forward Deployed Engineer roadmap. New to the role? Start with what a Forward Deployed Engineer does.