Language-model training and adaptation · MLD-1082
Scheduled sampling
A training scheme mixing reference and model-generated previous inputs over time.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Reduce dependence on perfect histories in sequence prediction.
Original caveat
It can introduce objective-consistency problems.
Context and aliases
Level: Intermediate. Stages: Training · Adaptation · Evaluation.
Tags: DL · SLM · LLM · NLP.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Transformers glossary ↗ · Consulted
- PEFT: LoRA conceptual guide ↗ · Consulted
- TRL: supervised fine-tuning ↗ · Reference link
- TRL documentation ↗ · Consulted
- Original reference URL ↗