Trees and ensembles · MLD-0476
Gradient boosting
Building an additive model by fitting new learners to objective-gradient information.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Fit tabular regression with shallow trees.
Original caveat
Learning rate, depth, and number of estimators interact.
Context and aliases
Level: Intermediate. Stages: Training · Inference · Evaluation.
Tags: ML.
Aliases: GBM · gradient boosting machine.
Related concepts
Concepts linking here
Technology connections
- AutoGluon — AutoML toolkit covering tabular and other supported prediction tasks.
- CatBoost — Gradient boosting with dedicated categorical-feature handling.
- FLAML — Resource-aware AutoML and optimization library.
- H2O-3 — Distributed platform for classical ML and AutoML.
- LightGBM — Histogram-based gradient-boosted tree framework.
- River — Online machine-learning library for streaming observations.
- scikit-learn — Consistent estimators, transforms and evaluation utilities for classical ML.
- statsmodels — Statistical estimation, tests and model diagnostics in Python.
- XGBoost — Optimized gradient-boosted decision-tree library.
Original references
- scikit-learn: ensemble methods ↗ · Reference link
- scikit-learn: decision trees ↗ · Reference link
- Original reference URL ↗