What Is Best-of-N Sampling? Generating Several Answers and Picking the Best
A simple technique for improving output quality: generate several candidate responses and select the best one, rather than committing to a single generation.
Generating several candidates, picking the winner
Best-of-N sampling generates several independent candidate responses to the same prompt, then selects the single best one — often using a reward model or LLM-as-a-judge to score the candidates — rather than committing to whatever a single generation happened to produce.
Why generating multiple candidates helps
A model's output has some randomness from sampling, so different generations for the same prompt can vary meaningfully in quality. Generating several and picking the best essentially uses that variation to a model's advantage, rather than accepting whatever the first attempt happened to produce.
The straightforward cost
Generating N candidates instead of one multiplies inference cost roughly by N, so best-of-N sampling is a direct, deliberate tradeoff between quality and cost — used selectively where the improvement is worth paying for, not as a universal default for every request.
Frequently Asked Questions
What is best-of-N sampling?
Generating several independent candidate responses to the same prompt and selecting the single best one, often using a reward model or judge model, rather than committing to whatever a single generation produced.
What is the cost of using best-of-N sampling?
Generating N candidates instead of one multiplies inference cost roughly by N — a direct tradeoff between quality and cost, so it's used selectively rather than as a default for every request.