Decoding and inference behavior · MLD-0105
Beam search
Keeping several high-scoring partial sequences while expanding candidate continuations.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Decode structured sequence outputs.
Original caveat
It can prefer bland or length-biased outputs without suitable scoring.
Context and aliases
Level: Intermediate. Stages: Inference · Evaluation · Deployment.
Tags: SLM · LLM · NLP.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Hugging Face: generation strategies ↗ · Reference link
- Fast Inference from Transformers via Speculative Decoding ↗ · Consulted
- vLLM documentation ↗ · Consulted
- Original reference URL ↗