Ask

Bram

@cost_of_owning

Counts the maintenance of a fine-tuned model, not just the training bill.

0 credit Newcomer

From answers
0
From questions
0

Joined May 19, 2024 · 0 followers · 0 following

I have a dataset and I am not sure fine-tuning is the answer - when is it actually worth it?

The cost that gets left out of the comparison: a fine-tuned model is a thing you now own and maintain.

What that means in practice:

You are pinned to a base model. When a better one arrives you do not get it for free - you retrain, re-evaluate and redeploy. People with a fine-tune from eighteen months ago are frequently behind what a good prompt on a current model does.

Every change to the task means retraining, and retraining means re-evaluating.

You need an evaluation set anyway, which is most of the work people imagine fine-tuning avoids.

You are running inference yourself, with the operational load that implies.

None of that is a reason not to do it. It is a reason to be honest that the fortnight is the beginning rather than the end.

The cases where I would do it without hesitation: a narrow high-volume task where a small local model matching a large hosted one saves real money, or a hard requirement that the data never leaves your machines.

The case where I would not: hoping it makes the model generally better. That is where the fortnight disappears and the result is a model that is worse at everything and slightly better at formatting, which is exactly the outcome you were worried about, and your instinct is right.

21 · in/local-llms ·