Reproduce “FLAT: Feedforward Latent Triangle Splatting for Geometrically Accurate Scene Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Orest Kupyn, Goutam Bhat, Philipp Henzler, Fabian Manhardt, Christian Rupprecht, Federico Tombari

FLAT predicts explicit triangle splats from one image using a video-diffusion prior and feedforward scene decoder. Ray-aligned parameterization and a product window support training. Across RealEstate10K and DL3DV, triangles score 0.853 cosine similarity on rendered normals versus 0.587 for 2D Gaussian splats. However, supervision differs, triangles are not best for pixel fidelity, and optional refinement remains necessary for opaque geometry.

Key insights

  • Explicit, non-volumetric surface primitives (triangle splats) can be directly and stably decoded from compressed video diffusion latents in a single forward pass, overcoming previous challenges with flat primitive prediction.
  • Careful parameterization, including ray-centered local frames and residual rotations, along with an improved differentiable rendering technique like the product window function, is crucial for stable training and effective gradient flow for non-volumetric decoders.
  • Triangle splat representations inherently achieve higher geometric accuracy and enable more robust opaque mesh conversion compared to volumetric 3D Gaussian Splatting, making them more suitable for applications requiring explicit surface assets.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.