Optimization and training dynamics · MLD-0099
Batch size
The number of examples or tokens grouped for a specified computation or update.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Choose an effective batch of 128 examples.
Original caveat
Specify per-device, global, token, or effective batch size.
Context and aliases
Level: Foundational. Stages: Training · Theory.
Tags: ML · DL · SLM · LLM.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Deep Learning: online textbook ↗ · Consulted
- PyTorch optimization algorithms ↗ · Reference link
- PyTorch documentation ↗ · Reference link
- Original reference URL ↗