Reproduce “Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions” on OpenResearch

About this paper

Hansika Hewamalage, Christoph Bergmeir, Kasun Bandara

This study from Monash University empirically evaluates various Recurrent Neural Network (RNN) architectures for univariate time series forecasting, comparing their performance against traditional statistical benchmarks like ETS and ARIMA. The research identifies optimal RNN configurations, provides practical guidelines for data preprocessing and hyperparameter tuning, and releases an open-source framework to facilitate the adoption of RNNs in forecasting.

Key insights

• Prior deseasonalization via STL Decomposition is crucial for RNNs when time series exhibit heterogeneous seasonal patterns or varying lengths; otherwise, RNNs can directly model homogeneous seasonality.
• The Stacked RNN architecture generally performs best, followed by Sequence to Sequence with a Dense Layer (S2SD), while the traditional autoregressive Sequence to Sequence (S2S) architecture performs worst due to error accumulation.
• LSTM cells with peephole connections and the self-tuning COCOB optimizer generally provide the best performance, with COCOB eliminating the need for manual learning rate tuning.

Summary by alphaXiv.