The Machine Learning “Advent Calendar” Day 13: LASSO and Ridge Regression in Excel

Regularization techniques like Ridge and Lasso regression address issues with collinearity in linear models by adding a penalty term to the loss function. This penalization limits the size of the coefficients, preventing them from becoming excessively large and unstable. Ridge regression (L2 penalty) shrinks coefficients towards zero, while Lasso regression (L1 penalty) can force some coefficients to become exactly zero, effectively performing feature selection. The content demonstrates how these methods do not change the underlying model structure but alter the training process by modifying the gradient descent update rule, which is illustrated with examples in Excel.