Our failover fixed the outage, then started answering invoice questions without the invoice

Send a scanned invoice to a document API and you get back a vendor, a total and a due date. There is a version of that where the model answering was never shown the invoice. The response looks identical to a correct one: same shape, same fields, marked successful, and charged, because the endpoint bills when a call succeeds.

Our routing was one well-meant change away from producing exactly that.

The key was set, and the key was dead

Three model providers sit behind one interface, and the model name on the request picks which one serves it. Each provider had a health check, and the check was: is the key present, and does it not begin with PLACEHOLDER.

On 2026-08-16 that check reported everything healthy while the public API was down. The shared Google credential had been revoked along with the cloud project it belonged to on 2026-08-01. The variable was still set, so it still read fine. Tested against the providers with the exact values in production: the Google key returned 400 API_KEY_INVALID, the Anthropic key returned 200, the OpenAI key returned 200. The invoice preset names a Google model, so /v1/invoice answered 503 with its log line reading provider "stub", model gemini-2.5-pro, while two working providers sat beside it untouched.

Looking at a key cannot tell you it was revoked. Only a call finds out. So the router stopped trusting presence and started trying the next provider when one actually failed, which is the whole idea of failover.

The retry made it worse

Four hours later, measured with a file attached: a PDF-bearing call hopped to Anthropic, then to OpenAI, two real provider round trips at 175ms and 148ms, ending at OpenAI.

Neither of those drivers ever looked at the attachment. Both build their request from the prompt and the message history alone. The document was dropped without a word, and the model was asked to "Extract the attached invoice." having been shown nothing. With live keys that comes back successful, and success is the thing that triggers the charge. A confident invention on a paid endpoint is worse than the error it replaced, because the error is visible and the invention is not.

Ask the driver, not the router

The first repair hardcoded "attachment means Google or nothing" into the router. That was true for about a week, for one reason: Google's was the only driver that read files. The version that survived asks the provider instead.

// claude.ts — every attachment must be a format this rail reads
supportsFiles(files: InferenceFile[]): boolean {
  return files.every(
    (f) => DOCUMENT_MIMES.has(f.mimeType) || IMAGE_MIMES.has(f.mimeType),
  );
}

// the router: a candidate must be able to SERVE the call, not merely hold a key
const usable = ordered.filter(
  (d) => d.isConfigured() && (!files?.length || d.supportsFiles(files)),
);

It takes the files rather than answering a yes/no flag, because reading is per format, not per provider.

Anthropic's driver then learned to send native document and image blocks, so a customer PDF has a real second option for the first time.

The rail that can read it is not always the rail you may use

The replacement Google credential is a free-tier AI Studio key, and Google's free tier is content used to improve their products. ParseRail's security page says the documents you send "are not retained as training data or kept for our own use". One invoice crossing that rail makes the published sentence false, so this product carries no Google credential at all.

Which leaves audio with nowhere to go. /v1/transcribe now carries unavailable: { since: "2026-08-19", reason } in the endpoint catalog, the request handler reads that same entry before doing any work and returns the reason verbatim, and the endpoint browser badges it Paused.

This is how we built ParseRail: https://parserail.kynth.studio/?utm_source=kynth-devto&utm_medium=social&utm_campaign=kynth

One shipped product, taken apart, once a month. What it does, what it cost to build, what the pipeline behind it looks like, and what the numbers did — read off the repository and the live site, not written from memory. Join the list.