What Is a Small Language Model (SLM)?
Not every AI task needs a massive frontier model — small language models trade some raw capability for speed, cost, and the ability to run on modest hardware.
Smaller, on purpose
A small language model (SLM) has dramatically fewer parameters than a frontier model — often in the millions to low billions, rather than tens or hundreds of billions — trading some general capability for much lower compute cost, faster responses, and the ability to run on a laptop, phone, or other modest hardware.
Where the gap closes
For narrow, well-defined tasks — classification, simple extraction, a specific chat use case — a well-trained small model can perform close to a much larger general-purpose one, especially after fine-tuning it specifically for that task rather than expecting broad, general capability.
Why they matter beyond cost
Running inference on-device rather than through a cloud API also matters for privacy (data never leaves the device) and offline use. This is a large part of why quantization and small language models are usually discussed together — both are aimed at making capable AI run somewhere other than a data center.
Frequently Asked Questions
What counts as a "small" language model?
There's no fixed cutoff, but SLMs are generally in the millions-to-low-billions parameter range, dramatically smaller than frontier models with tens or hundreds of billions of parameters.
Can a small language model be as good as a large one?
For narrow, well-defined tasks, a well-trained (and often fine-tuned) small model can come close to matching a much larger general-purpose model — the gap is widest on broad, open-ended tasks.