agent called search_docs 38 times in one run and burned $4.80 before max_steps agent-loop

The loop is doing exactly what you built. Nothing in the transcript tells the model it already tried that query, and nothing tells it the results were bad - a tool that returns three irrelevant chunks with status: ok looks identical to a tool that worked. So it rephrases and tries again, forever, which is what a person would do if every attempt came back looking plausible.

Four changes, cheapest first:

1. Dedupe at the tool boundary. Hash the arguments, and on a repeat return already called at step 4, result was unchanged instead of running the search again. Costs nothing and kills nine of your 38 immediately.
2. Make failure legible. Return the retrieval scores and say so: 3 results, top score 0.31, below threshold 0.6. A model that can see it is failing will stop; a model that cannot, will not.
3. Give it a way out. Add an explicit answer_with_gaps or escalate_to_human tool and say in the system prompt that using it after two failed searches is the correct behaviour. Without a legitimate exit, the only action available is another search.
4. Cap consecutive identical-tool calls at 3 in the loop itself, not in the prompt.

The real fix underneath all of that is retrieval. If step 1 had returned the refund policy, none of this happens.

On the $4.80: that is not 38 calls at 12 cents, it is the transcript being resent every step and growing each time. Step 38 pays for everything that happened in steps 1 to 37. Cost goes up roughly with the square of step count, which is why a runaway loop is so much more expensive than it looks.

Two mitigations that are independent of fixing the loop: turn on prompt caching so the stable prefix is not repriced every step, and truncate tool results aggressively - three chunks at 400 tokens each, repeated 38 times, is most of your bill and none of your answer quality.

Put a hard cost ceiling in the loop, not just a step ceiling. We track cumulative tokens per run and abort at a threshold with a specific message the user sees. Steps are a bad proxy for spend because one step can read a 60k token file and blow through the budget while your step counter says 4.

Also log every run's step count as a metric. A p99 that quietly climbs from 6 to 20 over a month is your retrieval degrading, and you will never notice it from average cost alone.

Before rearchitecting anything, go read those three chunks it kept getting back. In our version of this bug the retrieval was fine and the refund policy simply was not in the corpus - it lived in a PDF nobody had indexed. The agent was behaving reasonably about a document that did not exist, and every fix we tried on the loop was treating a symptom.

Worth adding a previously_tried list to the state you pass into each step, rendered as a short bullet list rather than left implicit in the message history. Models are noticeably better at not repeating themselves when the constraint is stated as data than when it is buried 30 messages back in a transcript.

38 attempts, no error, confident apology at the end. It has learned more from us than we intended.