Budgets that actually hold: stopping runaway LLM spend
Alerts tell you about overspend after it happens. Here's how prepaid balances, pre-authorization and hard caps make a runaway LLM bill structurally impossible.
Most teams discover their LLM spend the same way: a billing alert, three days late, after a retry loop quietly burned through a month of budget in an afternoon. Alerting is not control. This post is about the difference — and how to make overspend something that cannot happen, rather than something you find out about.
Why usage-based billing bites
Token pricing is elastic in exactly the wrong direction. A bad deploy, an infinite agent loop, or a prompt that balloons context can multiply cost by 100× without changing your request rate at all. The usual guardrails don’t help:
- Email alerts fire after the spend already happened.
- Rate limits cap requests per second, but cost and rate are unrelated — one expensive call can cost more than a thousand cheap ones.
- Per-provider dashboards each show a slice, so no one sees the total until the invoices land.
A budget you can’t enforce is just a wish with a dashboard.
Three controls that compose
Open Vertex Router treats the budget as part of the request path, not a report you read later. Three mechanisms stack:
1. Prepaid balance
You fund a balance up front. Spend draws it down in real time, metered per token as each response completes. When the balance is gone, it’s gone — there’s no open-ended invoice growing in the background.
2. Pre-authorization
Before a call is forwarded to a provider, Open Vertex Router estimates its maximum cost and checks it against the remaining balance and the key’s cap. If it wouldn’t fit, the request is rejected with a clear error before any tokens are spent.
3. Hard caps and overdraft
Each key, project and team carries an explicit limit plus an optional, bounded overdraft. The overdraft absorbs honest spikes; the hard cap guarantees the ceiling. There is no configuration in which a key silently exceeds its limit.
Setting a cap in practice
Caps are attached to keys. Create one with a monthly limit and a small overdraft:
curl -X POST https://api.openvertexrouter.com/v1/keys \
-H "Authorization: Bearer $OVR_ADMIN_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "prod-web",
"budget": { "limit_usd": 500, "period": "monthly" },
"overdraft_usd": 50
}'Your application code doesn’t change — it’s still a drop-in OpenAI client. The only difference is the base URL and the key you just scoped:
from openai import OpenAI
client = OpenAI(
base_url="https://api.openvertexrouter.com/v1",
api_key="ovr_sk_live_prod_web",
)
# If this call would breach the cap, it fails fast with a 402 — no spend.
resp = client.chat.completions.create(
model="anthropic/claude-sonnet-5",
messages=[{"role": "user", "content": "Summarize today's incidents."}],
)When a request can’t be authorized, you get a 402 Payment Required with the
remaining balance, so your app can degrade gracefully — queue the work, fall
back to a cheaper model, or surface a clear message.
Alerting vs. enforcement
The distinction is the whole point:
| Approach | When you find out | Can it overspend? |
|---|---|---|
| Email alerts | After the fact | Yes |
| Rate limits | At the limit | Yes (cost ≠ rate) |
| Pre-authorization | Before the call | No |
Where to go next
Budgets are one key away. Scope a key per environment, give production a real limit, and let staging run on a tiny overdraft so a bad test can’t cost more than lunch. The full reference — periods, rollover, team hierarchies — lives in the budget documentation.
The goal isn’t to watch spend more closely. It’s to make the worst case boring.