What Is a Training Epoch? One Complete Pass Through the Data
A basic unit for measuring training progress: one full pass through the entire training dataset, though large language models often don't train in whole epochs at all.
One full pass through the dataset
A training epoch is one complete pass through the entire training dataset — every example in the dataset gets used exactly once per epoch — a basic, commonly used unit for measuring and describing how much total training has happened.
Why more epochs isn't automatically better
Training for many additional epochs on the same fixed dataset increases the risk of overfitting, since the model sees the exact same examples repeatedly — the right number of epochs is a genuine tuning decision, typically guided by watching validation loss rather than picking an arbitrarily large number.
Why large language models often don't train in clean, whole epochs
Modern LLM pretraining corpora are so enormous that many models train for less than a single full epoch — seeing most training examples only once, or even less than once — rather than the traditional pattern of cycling through a comparatively smaller dataset many times over.
Frequently Asked Questions
What is a training epoch?
One complete pass through the entire training dataset, where every example gets used exactly once — a basic unit for measuring and describing how much total training has occurred.
Do large language models train for many epochs?
Often not — modern pretraining corpora are so enormous that many models train for less than a single full epoch, seeing most examples only once, rather than cycling through the data many times.