The distinction that decides it: fine-tuning changes how a model responds. It is a poor way to change what it knows.
So it works well for:
Format and structure. Always producing a particular shape, in a particular style, without being asked each time.
Tone and voice. Consistently sounding like your product rather than like a general assistant.
A narrow task done repeatedly, where you want a small fast model to match a large one on that one thing.
Reducing prompt size, which is your cost problem - behaviour baked in is behaviour you stop sending.
It works badly for:
Teaching facts. Training on documents does not reliably make a model able to recall them, and it will confidently produce plausible variations. Retrieval is the tool for knowledge, not training.
Anything that changes often. Retraining is not an edit.
Improving general reasoning. A small model fine-tuned on your data is still a small model.
So the question to ask about your two thousand examples: are they teaching a shape or teaching content? If the model already can do the task and is inconsistent about how, fine-tuning is the right tool. If it is getting the substance wrong, it is not.