What Is LoRA Fine-Tuning? Adapting a Model Without Retraining Everything
A technique that fine-tunes a model by training a small set of new parameters instead of touching the original ones — dramatically cheaper than full fine-tuning.
Training a small add-on instead of the whole model
LoRA (low-rank adaptation) fine-tunes a model by freezing its original parameters entirely and training a much smaller set of new, additional parameters that get combined with the original ones at inference time — rather than updating the full set of original weights directly.
Why training so much smaller a set still works
The insight behind LoRA is that the actual change needed to adapt a model for a new task tends to be far simpler than the full complexity of the original model's parameters — so a small, carefully structured set of new parameters can capture most of the useful adaptation, without needing to touch billions of original values.
Why this matters practically
Because LoRA trains a tiny fraction of the parameters a full fine-tune would touch, it needs far less memory and compute, and produces a small, portable file that can be applied on top of the original base model — making it practical to fine-tune large models on much more modest hardware than full fine-tuning would require.
Frequently Asked Questions
What is LoRA fine-tuning?
A fine-tuning technique that freezes a model's original parameters and trains a much smaller set of new, additional parameters that get combined with the original ones at inference time, rather than updating the full model directly.
Why is LoRA cheaper than full fine-tuning?
It trains only a small fraction of the parameters a full fine-tune would touch, needing far less memory and compute, and produces a small, portable file that layers on top of the unchanged base model.