Fine-tuning and prompting are two different ways to adapt a language model’s behaviour for a specific task. The decision between them is one of the most consequential architectural choices in building AI-powered products — and one that is frequently made based on intuition rather than analysis. The correct choice depends on what problem you are actually solving.
What Prompting Achieves
Prompting (providing instructions, examples, and context in the input) is the correct starting point for nearly all tasks. A well-engineered prompt can achieve: change in tone and style (“respond as a formal British professional”); task instruction (“classify this text as positive, negative, or neutral”); few-shot learning (providing 3–5 examples of input/output pairs teaches the model the format and pattern almost instantly); persona and constraint application (“you are a customer service agent for X company; do not discuss competitors”); knowledge injection (providing context the model does not have — a database excerpt, a policy document, a user’s account history). The advantage of prompting: zero setup cost, instant iteration (you change a prompt in seconds, not days), no risk of degrading the model’s general capabilities, and all improvements transfer to new model versions automatically. When prompting fails: tasks requiring knowledge the model genuinely lacks (events after training cutoff — prompting cannot fix this; retrieval augmented generation can); tasks requiring very specific output formats that the model consistently fails to produce despite instructions; tasks where consistent persona maintenance across thousands of turns is critical and prompting alone drifts; tasks where latency matters and a long system prompt adds too much to input token count.
When Fine-Tuning Actually Helps
Fine-tuning adjusts the model weights on a curated dataset of input/output pairs — the model’s internal parameters change. The cases where fine-tuning provides genuine value: format and structure consistency: if you need JSON output in a very specific schema and the model consistently produces variations despite detailed prompting, fine-tuning on examples is more reliable than prompt engineering; style and voice: fine-tuning on a specific writing style (a brand’s content, a legal document format, a domain-specific technical style) can achieve consistency that prompting struggles with; efficiency: a fine-tuned smaller model (e.g., Haiku 4.5) can match a much larger model (Sonnet, Opus) on a specific task — significant cost and latency savings at scale; reducing prompt length: if your system prompt is 5,000 tokens of instructions and examples, fine-tuning can encode that knowledge into the weights, allowing a shorter prompt; task-specific reasoning patterns: for domain-specific tasks (medical coding, legal document classification, specialised code generation) where the pattern differs fundamentally from the model’s general training distribution. What fine-tuning cannot fix: factual knowledge gaps (fine-tuning on facts does not reliably add knowledge — it often induces hallucination); general reasoning degradation (fine-tuning on a narrow task can reduce performance on adjacent tasks — the alignment tax); safety issues (fine-tuning on a task does not improve safety alignment; if safety is a concern, use a system prompt with the fine-tuned model). The practical recommendation: start with prompting; measure performance on your specific task; fine-tune only if prompting reaches a ceiling you cannot prompt-engineer past. Most production AI applications that jump to fine-tuning immediately would have performed better by investing the same engineering time in better prompts and retrieval.
