Reproduce “SkelSplat: Robust Multi-view 3D Human Pose Estimation with Differentiable Gaussian Rendering” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Laura Bragagnolo, Leonardo Barcellona, Stefano Ghidoni

SkelSplat refines a shared 3D skeleton by representing joints as Gaussian blobs, projecting them into calibrated camera views, and matching rendered joint-specific heatmaps to 2D detections. It requires synchronized views, an initial 3D pose, and a pose detector. On CMU-to-Human3.6M, it reports 20.3 mm versus 38.9 mm, with a target-fine-tuned detector; this is not a fully training-free pipeline.

Key insights

  • Gaussian Splatting can be effectively adapted from dense scene reconstruction to sparse, articulated structures like human skeletons for precise joint localization.
  • A one-hot encoding scheme and modified rendering function allow for independent optimization of individual human joints, even with overlapping 2D projections.
  • Robust multi-view 3D human pose estimation can be achieved via an optimization-driven approach, relying only on 2D keypoint detections, which inherently improves generalization and reduces 3D ground-truth dependency.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.