Regularization and generalization · MLD-0541
Implicit regularization
A preference for certain solutions caused by optimization or architecture without an explicit penalty.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
SGD favors some interpolating solutions over others.
Original caveat
The mechanism depends on the training setting.
Context and aliases
Level: Intermediate. Stages: Theory · Training · Evaluation.
Tags: ML · DL · SLM · LLM.
Aliases: None recorded.
Related concepts
Original references
- Deep Learning: online textbook ↗ · Consulted
- Google Machine Learning Glossary ↗ · Consulted
- PyTorch neural-network layers ↗ · Reference link
- Original reference URL ↗