What Is Dropout in Neural Networks? Randomly Disabling Parts of the Network
A surprisingly effective regularization technique: randomly turn off a fraction of a network's units during each training step, forcing more robust learning.
Randomly disabling units during training
Dropout randomly disables a fraction of a network's units during each individual training step — different units get disabled each time — forcing the rest of the network to work without being able to rely on any specific unit always being present.
Why randomly breaking the network like this actually helps
Without dropout, a network can develop an over-reliance on a small number of specific units doing most of the work, with the rest becoming somewhat redundant. Forcing the network to function well even when random units disappear pushes every part of it to learn more independently useful, robust representations, rather than a few units becoming critical single points of failure.
Where dropout is and isn't applied
Dropout is applied only during training — at inference time, the full network runs with every unit active, typically with a small adjustment to compensate for that difference. It's one of several regularization techniques used together, rather than a complete standalone solution to overfitting on its own.
Frequently Asked Questions
What is dropout in neural networks?
A regularization technique that randomly disables a fraction of a network's units during each individual training step, forcing the rest of the network to function well without relying on any specific unit always being present.
Is dropout used at inference time too?
No — dropout is applied only during training. At inference time, the full network runs with every unit active, typically with a small adjustment made to compensate for that difference.