Training
What Is Supervised Fine-Tuning (SFT)?
The training stage right after pretraining, where a model learns from curated example conversations what a good response actually looks like.
The stage after pretraining
A freshly pretrained model is good at continuing text plausibly but has no real sense of "answer this like a helpful assistant would." Supervised fine-tuning (SFT) is the next stage: training the model further on a curated set of high-quality example conversations — a prompt paired with the ideal response — so it learns the pattern of behaving like an assistant.
Where the training examples come from
SFT datasets are typically written or reviewed by humans specifically for this purpose — real questions paired with carefully composed ideal answers, covering the range of tasks and tones the model should be able to handle. Quality here matters enormously more than volume; a smaller set of excellent examples generally beats a huge set of mediocre ones.
SFT vs. what comes after
SFT gets a model to a reasonably good baseline of assistant-like behavior, but it's usually followed by a further refinement stage like RLHF or DPO, which sharpens the model's behavior further using comparative feedback rather than fixed example answers.
Frequently Asked Questions
What is supervised fine-tuning?
A training stage after pretraining where a model learns from curated example conversations — a prompt paired with an ideal response — to behave like a helpful assistant rather than a raw text predictor.
What comes after supervised fine-tuning?
Usually a further refinement stage like RLHF or DPO, which uses comparative feedback (which of two responses is better) rather than fixed example answers to sharpen the model's behavior.