Reproduce “Simple Contrastive Representation Learning for Time Series Forecasting” on OpenResearch

About this paper

Xiaochen Zheng, Xingyu Chen, Manuel Schürch, Amina Mollaysa, Ahmed Allam, Michael Krauthammer

SimTS presents a representation learning framework for time series forecasting that predicts future latent states from historical ones, eliminating the need for negative pairs. It achieved average improvements of 18.8% (MSE) and 11.1% (MAE) over TS2Vec across six benchmark datasets, demonstrating enhanced generalizability.

Key insights

• Explicit negative pairs, commonly used in contrastive learning, can degrade time series forecasting performance due to the difficulty of accurate construction and potential 'false repulsion' in temporal data.
• The stop-gradient mechanism is critical for preventing model collapse in a negative-pair-free representation learning setup, effectively guiding the encoder to learn predictive historical representations.
• Assumptions like distinct seasonal-trend components or universally effective data augmentations for time series often limit generalizability, suggesting simpler, assumption-free approaches can be more robust.

Summary by alphaXiv.