Reproduce “ADG: Ambient Diffusion-Guided Dataset Recovery for Corruption-Robust Offline Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zeyuan Liu, Zhihe Yang, Jiawei Xu, Rui Yang, Jiafei Lyu, Baoxiang Wang, Yunjian Xu, Xiu Li
Researchers from Tsinghua University and The Chinese University of Hong Kong developed Ambient Diffusion-Guided Dataset Recovery (ADG), a three-stage diffusion-based framework, to restore high-quality data from partially corrupted offline reinforcement learning datasets. ADG significantly improved performance across various RL algorithms, boosting MDP-based methods by an average of 69.1% and sequence modeling methods by 17.4% under random corruption, bringing scores close to those achieved with clean data.
Key insights
- The Ambient Diffusion concept can be theoretically extended to the discrete DDPM framework, enabling effective learning for noise prediction on partially corrupted data without assuming clean inputs.
- Decoupling the detection of corrupted samples from their denoising, using two distinct diffusion models, prevents interference and ensures the denoiser learns the true data distribution from clean examples.
- Incorporating temporal context via trajectory "slices" (parameter H) significantly enhances the performance of both the detector and denoiser, improving data recovery and subsequent RL algorithm scores.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.