Fine-tuning vs Prompting

When working with Large Language Models, developers face a fundamental choice: should they teach the model new behaviour through fine-tuning, or guide its existing knowledge through carefully crafted prompts? Prompting is the practice of writing instructions, examples, and context directly in the input text to steer the model towards a desired output. It requires no changes to model weights and works immediately - you can iterate on a prompt in seconds. Prompting is the right starting point for most use cases.

Fine-tuning involves taking a pre-trained LLM and continuing to train it on a smaller, task-specific dataset. This updates the model weights to make it consistently behave in a certain way - following a specific output format, adopting a brand voice, or mastering a domain-specific vocabulary. Fine-tuning is more expensive (it requires compute, data preparation, and training time) but produces more reliable results when prompting alone cannot achieve the consistency you need.

A good rule of thumb is: start with prompting, and only fine-tune when you have hundreds of high-quality examples and prompting has hit a ceiling. The below example shows how to use few-shot prompting to classify customer support tickets without any fine-tuning.

It gives the following output,

Ticket: My invoice shows the wrong amount
Category: BILLING

Ticket: The dashboard is not loading on Chrome
Category: TECHNICAL

Ticket: How do I reset my password?
Category: ACCOUNT

Ticket: Do you offer student discounts?
Category: OTHER

When prompting is not enough - for example when you need a model to always respond in a very specific JSON schema or use domain-specific jargon consistently - you can fine-tune the model. The below example shows how to prepare training data in the JSONL format required for fine-tuning a GPT model via the OpenAI API.

It gives the following output,

Training file created: training_data.jsonl
Total examples: 2

# After uploading and training (takes 10 to 30 minutes):
# Fine-tuning job ID: ftjob-abc123XYZ
# Status: succeeded
# Fine-tuned model: ft:gpt-4o-mini:your-org:custom-name:abc123

When to choose Prompting vs Fine-tuning:

Use Prompting when the task is general, you need fast iteration, you have few examples, or the use case changes frequently. Prompting is cheaper and requires no training time.

Use Fine-tuning when you need strict output format consistency, you have 100 or more high-quality labelled examples, the task requires specialised domain knowledge not in the base model, or you need lower latency by encoding instructions into weights rather than sending long system prompts every time.