The cost that gets left out of the comparison: a fine-tuned model is a thing you now own and maintain.
What that means in practice:
You are pinned to a base model. When a better one arrives you do not get it for free - you retrain, re-evaluate and redeploy. People with a fine-tune from eighteen months ago are frequently behind what a good prompt on a current model does.
Every change to the task means retraining, and retraining means re-evaluating.
You need an evaluation set anyway, which is most of the work people imagine fine-tuning avoids.
You are running inference yourself, with the operational load that implies.
None of that is a reason not to do it. It is a reason to be honest that the fortnight is the beginning rather than the end.
The cases where I would do it without hesitation: a narrow high-volume task where a small local model matching a large hosted one saves real money, or a hard requirement that the data never leaves your machines.
The case where I would not: hoping it makes the model generally better. That is where the fortnight disappears and the result is a model that is worse at everything and slightly better at formatting, which is exactly the outcome you were worried about, and your instinct is right.