Reproduce “Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Philipp Schröppel, Christopher Wewer, Jan Eric Lenssen, Eddy Ilg, Thomas Brox

Researchers developed Neural Point Cloud Diffusion (NPCD), the first diffusion model to achieve disentangled 3D shape and appearance generation through a hybrid neural point cloud representation. This method yields better generation quality than prior disentanglement-capable approaches and performs competitively with state-of-the-art methods that lack disentanglement features.

Key insights

  • The hybrid neural point cloud representation, with point positions for shape and point features for appearance, intrinsically supports disentangled 3D generation.
  • "Many-to-one mappings" in autodecoded latent spaces pose a challenge for diffusion models, which can be mitigated effectively through strategies like zero initialization and regularization (Total Variation, KL divergence).
  • The diffusion model can achieve disentangled generation by fixing one modality (shape or appearance) during the reverse diffusion process while generating the other.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.