Language-model training and adaptation · MLD-1101
Self-training
Iteratively training a model using labels or outputs generated by itself or a related model.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Use confident predictions to expand training data.
Original caveat
It can reinforce the model's own mistakes.
Context and aliases
Level: Intermediate. Stages: Training · Adaptation · Evaluation.
Tags: DL · SLM · LLM · NLP.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Transformers glossary ↗ · Consulted
- PEFT: LoRA conceptual guide ↗ · Consulted
- TRL: supervised fine-tuning ↗ · Reference link
- TRL documentation ↗ · Consulted
- Original reference URL ↗