Training
What Is Instruction Tuning? Teaching a Model to Follow Directions
Instruction tuning is what teaches a model to actually do what you ask, instead of just continuing your text in the most statistically likely direction.
The gap it closes
A raw, pretrained model will often continue a question with more questions in the same style, because "predict plausible next text" doesn't inherently include "recognize this as an instruction and follow it." Instruction tuning specifically trains a model on examples formatted as instruction-and-response pairs, so it learns to recognize and act on a request rather than just extend it.
How it's typically done
It's usually done via supervised fine-tuning on a large, diverse set of instructions paired with correct responses — covering many task types (summarizing, answering, coding, formatting) so the model generalizes the pattern "this is a request, respond to it directly" rather than memorizing specific instructions.
Why diversity of instructions matters most
A model instruction-tuned on only one kind of task (say, only question-answering) tends to struggle following instructions outside that pattern. The value of instruction tuning comes largely from the breadth of instruction types covered, not just the volume of examples.
Frequently Asked Questions
What is instruction tuning?
A training process that teaches a model to recognize and follow instructions, by training it on examples formatted as instruction-and-response pairs across a wide variety of task types.
Why doesn't a raw pretrained model already follow instructions?
Pretraining only teaches a model to predict plausible next text, not to recognize a prompt as a request to act on. Instruction tuning is what closes that specific gap.