Language-model training and adaptation · MLD-0399
Exposure bias
Mismatch between training on reference histories and generating from a model's own earlier outputs.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Errors accumulate during autoregressive generation.
Original caveat
The term names one mismatch, not every generation failure.
Context and aliases
Level: Intermediate. Stages: Training · Adaptation · Evaluation.
Tags: DL · SLM · LLM · NLP.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Transformers glossary ↗ · Consulted
- PEFT: LoRA conceptual guide ↗ · Consulted
- TRL: supervised fine-tuning ↗ · Reference link
- TRL documentation ↗ · Consulted
- Original reference URL ↗