What Is Parameter-Efficient Fine-Tuning (PEFT)?
The broader category of techniques that adapt a large model by training only a small fraction of its parameters — of which LoRA is the best-known example.
A category, not a single technique
Parameter-efficient fine-tuning (PEFT) is the general category of techniques that adapt a large pretrained model to a new task by training only a small fraction of its total parameters, rather than updating every single one — LoRA is the most widely used specific example, but not the only one.
The shared problem PEFT methods all address
Fully fine-tuning every parameter of a large model requires storing gradients and optimizer state for the entire model simultaneously, which demands enormous memory — PEFT methods share the same basic goal of avoiding that cost, each with a different specific mechanism for achieving it.
Other approaches in the same family
Beyond LoRA, the PEFT family includes adapter layers (small trainable modules inserted into the network), and prompt or prefix tuning (learning a small set of input-level parameters instead of touching the model's internal weights at all).
Frequently Asked Questions
What is parameter-efficient fine-tuning?
The general category of techniques that adapt a large pretrained model by training only a small fraction of its total parameters, avoiding the enormous memory cost of fully fine-tuning every parameter.
Is LoRA the same thing as PEFT?
LoRA is the most widely used specific technique within the broader PEFT category — PEFT also includes other approaches like adapter layers and prompt or prefix tuning, each with a different specific mechanism.