What Is a Scaling Law in AI? Predicting Performance Before Training
A set of empirical patterns showing that model performance improves predictably as data, parameters, and compute increase together — letting labs plan huge training runs in advance.
A predictable relationship between scale and performance
Scaling laws are empirical patterns showing that a model's performance improves in a fairly predictable way as its size, its training data, and the compute used to train it all increase together — not a guess, but a measurable, repeatable relationship observed across many training runs.
Why this predictability matters enormously in practice
A frontier training run costs an enormous amount, so being able to predict roughly how a much larger, more expensive run will perform — based on trends observed at smaller, cheaper scale — lets labs make informed decisions about whether a given scale-up is likely worth its cost before actually committing to it.
Scaling laws and compute-optimal training
Scaling laws are also what compute-optimal training research is built on — figuring out, for a given compute budget, the best split between spending on a larger model versus spending on more training data, rather than assuming bigger is automatically better in isolation.
Frequently Asked Questions
What is a scaling law in AI?
An empirical pattern showing that model performance improves in a predictable way as model size, training data, and compute all increase together — a measurable relationship observed across many training runs.
Why do scaling laws matter for training decisions?
They let labs predict how a much larger, more expensive training run is likely to perform based on trends observed at smaller scale, informing whether a given scale-up is worth its substantial cost before committing to it.