The Machine Learning “Advent Calendar” Day 21: Gradient Boosted Decision Tree Regressor in Excel

Gradient Boosting builds a powerful predictive model by sequentially adding simple decision trees to an ensemble. Each new tree is cleverly trained to predict the errors, or residuals, of the current model, identifying where the ensemble is weakest. The new tree's predictions are then scaled by a learning rate and added to the model, making gradual corrections that improve accuracy with each iteration. Through this additive process, the accumulation of many simple trees creates a final model that can approximate highly complex and non-linear patterns.