Why MLOps Retraining Schedules Fail — Models Don’t Forget, They Get Shocked

Many MLOps practices rely on scheduled retraining, assuming model performance decays smoothly and predictably over time. An experiment on a production-like fraud detection dataset demonstrates this assumption is often flawed, yielding a negative R² value when fitting a traditional exponential decay curve. The analysis reveals that performance doesn't decay gradually but instead drops suddenly in response to external events, a pattern termed the "episodic regime." A simple diagnostic using the R² value of an exponential fit can determine if a model's performance follows a smooth or episodic pattern. For episodic systems, calendar-based retraining should be replaced with shock-detection mechanisms that trigger retraining only when a significant performance drop occurs.