The Machine Learning “Advent Calendar” Day 16: Kernel Trick in Excel

The kernel trick in SVM is explained by building an analogy to Kernel Density Estimation, where a local function, or 'bell,' is centered on each data point. These individual functions, often Radial Basis Functions (RBFs), are then weighted and summed to create a complex, non-linear decision function. The 'SVM' part of the model comes from the hinge loss function, which has a unique property of forcing the weights of correctly classified points to zero. This results in a sparse model where only a few crucial data points, called support vectors, actively contribute to the final classification boundary, contrasting with non-sparse models like kernel ridge regression.