Reproduce “Diffusion Model-Augmented Behavioral Cloning” on OpenResearch

About this paper

Shang-Fu Chen, Hsiang-Chun Wang, Ming-Hao Hsu, Chun-Mao Lai, Shao-Hua Sun

Researchers from National Taiwan University developed Diffusion Model-Augmented Behavioral Cloning (DBC), an imitation learning framework that combines the inference efficiency of direct policy learning with the generalization capabilities of a diffusion model. This method consistently performs competitively or superiorly across various continuous control tasks while maintaining high inference speed and robustness against out-of-distribution states and manifold overfitting.

Key insights

• Combining the conditional probability modeling of BC with the joint probability modeling of a diffusion model provides complementary benefits, leveraging BC's efficiency and a diffusion model's generalization.
• A diffusion model can effectively augment a BC policy by providing a robust learning signal to match the expert's joint distribution, rather than serving directly as the policy itself.
• The two distinct loss functions ($L_{BC}$ and $L_{DM}$) approximate forward and reverse Kullback-Leibler (KL) divergences, respectively, promoting both mode coverage and high-quality sample generation.

Summary by alphaXiv.