Statistics and experimental design · MLD-0110
Benchmark
A standardized task, dataset, or protocol for comparing systems.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Evaluate multiple models under the same test procedure.
Original caveat
A benchmark score captures only the measured task.
Context and aliases
Level: Intermediate. Stages: Theory · Evaluation · Research.
Tags: ML · DL · SLM · LLM.
Aliases: None recorded.
Related concepts
Concepts linking here
Technology connections
- DeepEval — Testing framework for LLM applications and metrics.
- HELM — Framework for broad language-model evaluation.
- Hugging Face Evaluate — Common interfaces to evaluation modules and measurements.
- Inspect AI — Evaluation framework for language models and agents.
- lm-evaluation-harness — Standardized language-model evaluation framework.
- promptfoo — Configuration-driven prompt/model testing and red-teaming toolkit.
- pytest — Python testing framework.
- Ragas — Evaluation toolkit for retrieval and LLM pipelines.
- TorchMetrics — PyTorch-oriented metric implementations with distributed aggregation.
- TruLens — Instrumentation and evaluation tooling for AI applications.
Original references
- scikit-learn: cross-validation ↗ · Reference link
- Mathematics for Machine Learning ↗ · Reference link
- scikit-learn: evaluation metrics ↗ · Reference link
- Original reference URL ↗