Optimization and training dynamics · MLD-0104
Bayesian optimization
Sequential optimization using a surrogate model and an acquisition rule to choose evaluations.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Tune an expensive training pipeline.
Original caveat
Modeling and acquisition overhead may not suit cheap evaluations.
Context and aliases
Level: Intermediate. Stages: Training · Theory.
Tags: ML · DL · SLM · LLM.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Deep Learning: online textbook ↗ · Consulted
- PyTorch optimization algorithms ↗ · Reference link
- PyTorch documentation ↗ · Reference link
- Original reference URL ↗