Gradient-Boosting

one model is an opinion; a committee is an estimator. 𐃏 ensemble methods build many imperfect predictors and combine them, and the two great families attack opposite ends of the bias-variance decomposition: bagging averages low-bias, high-variance models to cancel their wobble; boosting stacks up high-bias, low-variance weak learners to build accuracy that none of them has alone.

Median house values for 20,640 California census block groups from the 1990 census. The modern default for “show me a real regression problem”: big enough to be non-trivial, small enough to fit anywhere, and full of instructive pathologies — capped targets, aggregate features, and spatial structure.

Provenance#

Constructed by R. Kelley Pace and Ronald Barry for Sparse Spatial Autoregressions, Statistics and Probability Letters 33(3), 1997 — the point of the paper was spatial statistics, not machine learning. The data derive from the 1990 US census at the block group level (the smallest census unit, typically 600 to 3,000 people). It was long distributed via CMU’s StatLib archive; sklearn’s fetch_california_housing mirrors that original. A cosmetically extended variant (with an ocean_proximity categorical) is the running example in Geron’s Hands-On Machine Learning, chapter 2 — numbers from the two variants are not interchangeable.