What Is Teacher Forcing? Training a Model on the Correct Sequence, Not Its Own Guesses
A standard training technique where a model is trained using the actual correct sequence so far, rather than its own — potentially wrong — previous predictions.
Training on the real sequence, not the model's own guesses
Teacher forcing trains a sequence-generating model by feeding it the actual, correct sequence so far at each training step — rather than the model's own previous predictions, which might already be wrong — so each training step is evaluated against a clean, correct context rather than one potentially already contaminated by an earlier mistake.
Why this makes training meaningfully more stable
Without teacher forcing, an early wrong prediction during training would feed forward into every subsequent step of that same example, compounding the error and making it much harder to isolate what specifically went wrong at each individual step — teacher forcing keeps each step's learning signal clean and stable, isolated from earlier mistakes.
The gap between training and real generation
The catch is that during actual generation (inference), a model has no access to a "correct" sequence — it only has its own, potentially imperfect previous outputs to build on. This mismatch between how a model trains and how it actually generates is a known, studied gap, though in practice it hasn't prevented teacher forcing from remaining the standard training approach.
Frequently Asked Questions
What is teacher forcing?
A training technique where a sequence-generating model is fed the actual correct sequence so far at each training step, rather than its own previous predictions, keeping each step's learning signal clean and isolated from earlier mistakes.
What is the downside of teacher forcing?
During actual generation, a model has no access to a "correct" sequence — only its own previous outputs — creating a known mismatch between how the model trains and how it actually generates, though it hasn't prevented teacher forcing from remaining standard practice.