24
Several tools shipped this month just to show what a coding-agent session costs. What actually burns the budget, and can I see it without adding a third-party service? Category watch
I look after the AI spend for a small team and this month I have counted more than one product whose entire pitch is "see what your agent sessions actually cost".
When a category appears that fast it usually means the underlying thing is genuinely not visible, so before I put another service in the middle of our workflow I would like to understand the problem properly.
What I know: we pay a predictable amount per seat. What I do not know: which sessions are expensive, why, and whether that is anybody's fault.
Specifically:
- In a long agent session, what is actually consuming the budget? My assumption is that it is not the code it writes but the context it re-sends on every step, and I would like to know if that is right.
- Is any of this visible from what the vendors already give you, or is a third-party collector genuinely the only way?
- If we did add one, what is it seeing? These sessions contain our source code.
Not looking to cut spend for its own sake. I want to be able to answer "why was last month higher" with something better than a shrug.
@switch_cost_sena · yesterday · 3 replies
Your assumption in question 1 is right and it is the whole answer.
An agent turn is not one request. It is a loop: read files, call a tool, get output, decide again. Every step re-sends the accumulated conversation, so cost grows with the square of the session length rather than in proportion to it. A session that ran forty tool calls did not cost forty times a single call — it cost considerably more.
What that means practically is that the expensive sessions are almost never the ambitious ones. They are the ones that went in circles: a failing test the agent kept re-reading, a large file pulled into context early and carried for the rest of the run, a loop of edit-and-check that never converged. The work product looks small, which is exactly why nobody suspects it.
So the metric you want is not cost per session. It is cost per session divided by whether anything got merged.
Reply
Report
@switch_cost_sena · yesterday
It changes it a lot, and it is the reason two teams with identical usage can get very different bills.
Re-sending a prefix that has not changed can be billed at a fraction of the normal rate, so a session that keeps a stable context is dramatically cheaper than one that keeps disturbing it. The thing that ruins it is anything that mutates the early part of the conversation — an injected timestamp, a reordered file list, a tool that rewrites its own instructions.
So "did the cache hit" is a genuinely useful number and it is one of the few where a tracking tool earns its place. It is also the number you can act on: the fix is usually a small change to how context is assembled, not a change of model.
Reply
Report
@token_budget_bex · yesterday
That reframes what I should be measuring. If it is loop length rather than task size, then the intervention is a cap on how long a session may go without producing something, not a smaller model.
Does caching change the arithmetic much, or is that mostly marketing?
Reply
Report