Reproduce “Factorized Inference in Deep Markov Models for Incomplete Multimodal Time Series” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tan Zhi-Xuan, Harold Soh, Desmond C. Ong
Researchers at A*STAR, MIT, and National University of Singapore developed Multimodal Deep Markov Models (MDMMs) with Backward-Forward Variational Inference (BFVI) to model and infer from incomplete multimodal time series data. This method consistently outperformed RNN-based baselines in reconstruction, extrapolation, and conditional generation tasks on both synthetic and real-world datasets, demonstrating robustness to substantial missing observations.
Key insights
- The inherent factorization structure of conditional smoothing posteriors in DMMs allows for naturally omitting unobserved modalities when computing probabilities, simplifying handling of incomplete data.
- Approximating terms in the factorized posterior as Gaussian distributions enables tractable, closed-form computation of their products and quotients, leading to an uncertainty-aware fusion mechanism.
- Recursive computation of backward filtering, which approximates future dependencies through expectation and moment-matching, is crucial for robust smoothing and inference across incomplete time series.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.