Statistics and experimental design · MLD-0789
Multiple testing
Performing several hypothesis tests, increasing opportunities for false discoveries without adjustment.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Compare many model variants on one benchmark.
Original caveat
Model and metric selection can create hidden multiplicity.
Context and aliases
Level: Intermediate. Stages: Theory · Evaluation · Research.
Tags: ML · DL · SLM · LLM.
Aliases: Multiple comparisons.
Related concepts
Concepts linking here
Original references
- scikit-learn: cross-validation ↗ · Reference link
- Mathematics for Machine Learning ↗ · Reference link
- scikit-learn: evaluation metrics ↗ · Reference link
- Original reference URL ↗