The debate over whether Transformers actually work for time series forecasting just got more interesting. A recent AAAI 2023 paper claimed they don’t—arguing that a simple linear model called DLinear outperforms Transformer-based approaches. The team behind the Informer model, which won AAAI 2021’s best paper award, pushed back with their own benchmarking. Their results show Autoformer beating DLinear on all three datasets used in the original comparison.
Autoformer, published at NeurIPS 2021 by Wu, Haixu, et al., takes a fundamentally different approach to sequence modeling. Rather than relying on standard self-attention, it introduces an auto-correlation mechanism that exploits period-based dependencies in the data. The model also brings classical time series decomposition—splitting data into trend and seasonal components—directly into the Transformer architecture through a dedicated decomposition layer.
The real issue with DLinear, according to the analysis, is capacity. A linear model can’t incorporate covariates the way a Transformer can in univariate settings. That’s a significant limitation for real-world forecasting tasks where external variables matter. The new Autoformer implementation is now available in Hugging Face Transformers, making it accessible to practitioners who want to test these claims themselves.
What’s striking here is how the debate mirrors earlier skepticism about Transformers in computer vision. Initial papers claimed simple models were competitive, but architectures evolved and the gap widened. The same pattern may be playing out in time series. The authors don’t mince words: DLinear’s advantage is situational, not fundamental.
