Reproduce “Sonnet: Spectral Operator Neural Network for Multivariable Time Series Forecasting” on OpenResearch

About this paper

Yuxuan Shu, Vasileios Lampos

Sonnet, developed at University College London, integrates learnable wavelet transformations, spectral coherence-based attention, and Koopman operator theory to improve multivariable time series forecasting. The model achieved superior accuracy, outperforming leading baselines by an average MAE reduction of 2.2% across 12 diverse datasets while maintaining computational efficiency.

Key insights

• Spectral coherence-based attention effectively captures inter-variable dependencies by focusing on shared frequency patterns, leading to substantial performance gains, especially in models lacking explicit dependency mechanisms.
• Decomposing time series into learnable wavelet components prior to attention allows for multi-resolution time-frequency analysis, preserving temporal structure while enabling robust dependency modeling.
• Integrating Koopman operator theory enables the model to learn stable, linear dynamics for nonlinear temporal evolution, which is crucial for reducing error accumulation in long-term forecasts.

Summary by alphaXiv.