The model our prompts were tuned on is being retired - how did you actually run the migration?

Diff first, always. You cannot rewrite prompts sensibly until you know which of them actually changed behaviour, and in my experience it is a minority - typically the long ones and the ones leaning on a specific output habit. Spend day one building the harness, day two on the handful that moved, day three on monitoring you should have had anyway.

We shadowed it: same requests to both, old one serving, new one logged, for about a week. Cost roughly double on that path for the week and was the cheapest information we bought all quarter, because it surfaced exactly the failure you are describing - silently missing fields on long documents, which no spot check would have caught. If your volume makes that expensive, shadow a sample rather than everything.

Before rewriting a single prompt, check whether constrained or structured output is available on the new version for your call. Half the migrations I have watched were people hand-tuning wording to coax out valid JSON when they could have made invalid JSON impossible instead. That also makes the next deprecation almost a non-event.

Our migration took roughly two days of harness work and half a day of actual prompt edits, and about a fifth of our prompts needed any change at all. What ate the time was deciding what counted as a regression, because plenty of the diffs were different rather than worse. Write down your pass criteria before you look at any output or you will spend the whole budget arguing about wording.

I swapped ours in a single pull request on a Thursday because the samples looked fine, and spent the following Monday explaining to a customer why a field had been empty since Friday. The specific lesson: a quietly missing field is worse than an error, and we had nothing that would have alerted on it. Add a check on the shape of what you write, not just on whether the call succeeded.