Reproduce “Scalable Transformer for High Dimensional Multivariate Time Series Forecasting” on OpenResearch

About this paper

Xin Zhou, Weiqing Wang, Wray Buntine, Shilin Qu, Abishek Sriramulu, Weicong Tan, Christoph Bergmeir

STHD (Scalable Transformer for High Dimensional multivariate time series), developed by researchers at Monash University, VinUniversity, and the University of Granada, addresses the challenges of capturing inter-series dependencies efficiently in high-dimensional multivariate time series forecasting. It consistently outperformed state-of-the-art baselines on three real-world datasets, demonstrating improved accuracy and training efficiency by selectively modeling relevant inter-channel relationships.

Key insights

• Channel-dependent models, despite their theoretical advantages, often underperform channel-independent models in high-dimensional time series forecasting due to noise from irrelevant series and computational bottlenecks.
• Sparsifying the relationships between time series by identifying and using only the most relevant series significantly reduces noise and computational complexity.
• Reshaping input data through a strategy like ReIndex can drastically reduce memory usage and improve training efficiency for high-dimensional time series, enabling effective processing by transformer architectures.

Summary by alphaXiv.