agent retry loop burned $612 in six days before my budget alert fired LLM Costs

The boring answer is that you keep your own ledger and stop reading the provider dashboard for anything except reconciliation.

Every model call returns a usage object. Write one row per call before you return the response:

request_id, user_id, feature, model,
input_tokens, cached_input_tokens, output_tokens,
attempt_no, ts

Cost is then a SQL query with a price table you own, not a dashboard. Two useful things fall out immediately: cost per user, and cost per feature. When I did this the first query showed 4% of accounts were 61% of spend, all of them on one code path.

Second thing: attempt_no in that table is the column that would have caught this on day one. Alert on p99 attempts, not on dollars. Dollars are a lagging indicator of a loop that is already broken.

Also, if you are re-sending the whole file every attempt, put the static part of the prompt first and use prompt caching. Cached input tokens are dramatically cheaper than fresh ones, and a retry loop is the single best-shaped workload for it because everything except the last message is identical.

Your bug is not the spend, it is an unbounded loop with a network call in it. You'd be equally upset if it was a database.

The control you want is a ceiling enforced synchronously in the request path. Increment a counter keyed by user and day, before the call, and refuse when it's over:

const spent = await kv.incrbyfloat(`spend:${uid}:${day}`, estCost)
if (spent > limits[plan]) throw new SpendCapExceeded()

Estimate before, correct after with the real usage numbers. It drifts a bit and that's fine.

Provider budget alerts are advisory and they lag. I have seen them arrive 14 hours after the money was gone. They exist so you can find out, not so you can prevent.

One thing nobody mentions until it hurts: log the inputs alongside the usage, not just the counts. When you find the expensive 3% of requests you want to be able to look at what they actually were. I keep a hash of the prompt and a truncated first 500 chars. Cost me about 40MB a month and saved a full day of guessing.

You set a monthly budget notification and expected it to behave like a circuit breaker. It is a notification. It notifies. That is the whole product.

Simplest fix is to swap the model for the cheap tier. It's something like 20x less per token, so your $612 becomes $30 and you can stop worrying about the loop entirely.

"in testing it always converged in two" is going on a t-shirt.