Reproduce “WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shunyu Wu, Jiawei Huang, Weibin Feng, Boxin Li, Xiao Zhang, Erli Meng, Dan Li, Jian Lou and 1 more
Researchers from Sun Yat-sen University, Xiaomi Corporation, and National University of Singapore developed WaveMoE, a wavelet-enhanced Mixture-of-Experts foundation model, to improve time series forecasting by explicitly integrating frequency-domain information. This model demonstrated superior performance on a majority of 16 benchmark datasets, achieving lower average MSE and MAE compared to baseline time series foundation models.
Key insights
- Explicitly incorporating wavelet-domain representations into foundation models provides a valuable inductive bias, enhancing the capture of complex patterns like periodicity and multi-scale dynamics.
- A dual-path Mixture-of-Experts architecture can effectively integrate and coordinate both time-domain and wavelet-domain information through a unified routing mechanism.
- Scaling pretraining corpora in the frequency domain, alongside temporal data, can lead to substantial improvements in the generalization and robustness of Time Series Foundation Models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.