Your Model Isn’t Done: Understanding and Fixing Model Drift

Model drift is the deterioration of a predictive model's performance over time after it has been deployed into production. This occurs because the real-world environment changes, leading to either data drift, where feature distributions shift, or concept drift, where the relationship between features and the target variable changes. Proactive monitoring of performance metrics and feature distributions is the best way to detect drift before it significantly impacts results and erodes stakeholder trust. Once detected, drift can be addressed by either fixing the data input pipeline to match the training format or by retraining the model on newer data. Managing model drift is a critical, ongoing task for data scientists to ensure a model has a lasting and reliable impact.