Reproduce “Incomplete Data, Complete Dynamics: A Diffusion Approach” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zihan Zhou, Chenguang Wang, Hongyi Ye, Yongtao Guan, Tianshu Yu
Researchers at The Chinese University of Hong Kong, Shenzhen, developed a diffusion-based framework that learns complete physical dynamics directly from inherently incomplete observations. This framework offers theoretical guarantees for recovering the true generative process and demonstrates superior imputation accuracy on synthetic PDE datasets and real-world climate data, especially in sparse and structured missing data regimes.
Key insights
- A model trained on incomplete data learns conditional expectations for a dimension only if its probability of being queried is non-zero, necessitating strategic context-query partitioning.
- Ensemble averaging effectively approximates the full conditional expectation for complete data reconstruction, mitigating the distribution mismatch between training and inference by reducing variance.
- Optimal performance is achieved with moderate context-query ratios, balancing the information gap between observed and missing data with the frequency of parameter updates for unobserved regions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.