What Is Beam Search? Generating Text by Weighing Several Options at Once
Instead of committing to one token at a time, beam search keeps several candidate sequences alive in parallel — a middle ground between greedy decoding and full randomness.
Keeping multiple candidates alive
Beam search keeps a fixed number of the most promising candidate sequences ("beams") at every generation step, expanding each one and keeping only the overall best-scoring set going forward, rather than committing to a single token immediately like greedy decoding does.
Why this can beat greedy decoding
Because it tracks several possibilities in parallel, beam search can recover from a token choice that looked slightly suboptimal in isolation but leads to a much stronger overall sequence — something pure greedy decoding, which never reconsiders, simply can't do.
Where it's used, and where it isn't
Beam search is common in tasks with a fairly well-defined "correct" output, like translation, where finding the single best sequence matters. It's less common in open-ended chat generation, where sampling-based methods that introduce controlled randomness tend to produce more natural, varied text.
Frequently Asked Questions
What is beam search?
A text generation strategy that keeps several of the most promising candidate sequences in play simultaneously at each step, rather than committing to a single token immediately — aiming to find a better overall sequence than greedy decoding.
Is beam search used in chatbots like ChatGPT?
Not typically for open-ended conversation — sampling-based methods usually produce more natural, varied output there. Beam search is more common in tasks with a clearer single best answer, like translation.