I Pitted XGBoost Against Logistic Regression on 358 Matches. The Boring Model Won.

An experiment comparing five machine learning classifiers on a small dataset of 358 soccer matches found that a simple logistic regression model outperformed more complex ones like XGBoost. The primary evaluation metric was log-loss, which heavily penalizes confident but incorrect predictions, and the simpler model won. This outcome is attributed to the bias-variance tradeoff, as the high-capacity models like XGBoost overfit the limited data, leading to poor calibration and higher error. The success of the linear model demonstrates the importance of matching model complexity to data size and problem structure, as its inherent assumptions were a better fit for the data.