Reproduce “DiffusionBrowser: Interactive Diffusion Previews via Multi-Branch Decoders” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Susung Hong, Chongjian Ge, Zhifei Zhang, Jui-Hsien Wang

DiffusionBrowser enables interactive, multi-modal previews during video diffusion by employing a multi-branch decoder to extract scene intrinsics like albedo, depth, and normals from early intermediate features. This framework generates previews in about 0.53 seconds, facilitating new user controls such as stochastic renoising and latent steering, and shows over 74% preference for content predictability in user studies.

Key insights

  • Scene intrinsic properties, especially depth and surface normals, emerge and become stable remarkably early (within 5-15% of timesteps) in the video diffusion model's denoising process.
  • Naive single-branch decoders suffer from a 'superposition problem' at low inference steps, producing blurred or averaged outputs when the underlying posterior distribution is multimodal.
  • A multi-branch architecture, trained with a combination of branch-wise and ensemble losses, can effectively disentangle multimodal distributions, leading to clearer, artifact-free previews.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.