What Is Elastic Weight Consolidation? Protecting Old Skills While Learning New Ones
A technique that specifically protects a model's important existing parameters while it learns a new task, directly targeting the risk of catastrophic forgetting.
Protecting the parameters that matter most for old skills
Elastic weight consolidation identifies which of a model's parameters were most important for a previously learned task, then specifically discourages those particular parameters from changing much during subsequent training on a new task — while leaving less critical parameters freer to adjust.
Why this directly targets a well-known failure mode
Training on a new task without any such protection risks catastrophic forgetting — the new training overwriting parameter patterns the old capability depended on. Elastic weight consolidation is a targeted defense specifically against that risk, rather than a general regularization technique applied uniformly.
The tradeoff involved
Protecting old-task-important parameters too heavily can limit how much a model is able to learn on the new task — there's a genuine balance between preserving old capability and allowing enough flexibility for real new learning to happen, which is what the technique's tuning parameters are meant to control.
Frequently Asked Questions
What is elastic weight consolidation?
A technique that identifies which model parameters were most important for a previously learned task, then specifically discourages those parameters from changing much during subsequent training on a new task.
What problem does elastic weight consolidation solve?
It directly targets catastrophic forgetting — the risk that training on a new task overwrites the parameter patterns an older, previously learned capability depended on.