Reproduce “Enhancing Continuous Time Series Modelling with a Latent ODE-LSTM Approach” on OpenResearch

About this paper

C. Coelho, M. Fernanda P. Costa, L.L. Ferrás

Researchers at the University of Minho and University of Porto developed Latent ODE-LSTM, a generative model that addresses vanishing and exploding gradients in continuous-time recurrent networks for irregularly sampled time series. The model demonstrates improved reconstruction and extrapolation on synthetic spirals, and comparable or enhanced long-term forecasting accuracy on real-world climate and stock data compared to Latent ODE-RNNs.

Key insights

• Latent ODE-LSTMs effectively mitigate vanishing gradients through their internal cell state and gating mechanisms, which enable a more stable error flow compared to simpler ODE-RNN structures.
• Despite mitigating vanishing gradients, theoretical analysis shows that Latent ODE-LSTMs remain susceptible to exploding gradients, highlighting the necessity of explicit control mechanisms like Norm Gradient Clipping.
• The architecture significantly improves the ability to reconstruct and extrapolate continuous-time series, especially with sparse data, by leveraging the long-term dependency learning strengths of LSTMs within a continuous domain.

Summary by alphaXiv.