# Routing without regret: a quality-aware fallback design

> The hardest part of a router isn't picking the cheapest model — it's deciding when to escalate. We open-sourced the eval harness we use internally.

- Source: https://spendtensor.com/blog/routing-without-regret
- Author: Spend Tensor
- Published: Jun 4, 2026
- Category: Engineering
- Reading time: 11 min
- Topics: Routing, Evals, Open Source

---

Every router is a bet that the cheap model is good enough. The interesting engineering is the loss function when you're wrong.

Our design has three layers: a cheap classifier picks a candidate model, a streaming quality probe scores the first 60 tokens against a calibrated rubric, and a fallback path swaps providers mid-stream if the score drops below threshold. End-to-end overhead is under 5ms p99.

We're publishing the eval harness under Apache 2.0 this week. It includes calibration data for 9 model families and a replay tool to A/B routing policies against historical traffic.

---

Published by SpendTensor — AI spend management and LLM FinOps. Quote with attribution and a link to https://spendtensor.com/blog/routing-without-regret.