The Machine Learning “Advent Calendar” Day 15: SVM in Excel

The Support Vector Machine (SVM) model is explained by building upon concepts from logistic regression and regularization, rather than starting with the traditional geometric explanation. It introduces the hinge loss function as an alternative to log-loss, which penalizes misclassified points and those too close to the decision boundary. By adding a regularization penalty to the hinge loss, the model selects a unique solution, and the points that define this boundary are known as support vectors. The training process uses gradient descent to optimize a cost function that balances model simplicity and classification accuracy. Finally, this optimization-based view is contrasted with the more common geometric formulation of maximizing the margin.