Timer-XL: A Long-Context Foundation Model for Time-Series Forecasting

Timer-XL is a decoder-only Transformer foundation model specialized for long-context time-series forecasting. It supports variable context lengths, prediction lengths, and exogenous variables within a single, unified framework. The model's design is based on the observation that decoder architectures currently outperform encoder-only models for forecasting tasks. A core innovation is "TimeAttention," a causal attention mechanism that enables the model to effectively process long sequences by focusing on recent data while selectively referencing important past information. This approach allows Timer-XL to handle longer lookback windows and achieve superior performance compared to other models.