Reproduce “PhysGS: Bayesian-Inferred Gaussian Splatting for Physical Property Estimation” on OpenResearch

About this paper

Samarth Chopra, Jing Liang, Gershom Seneviratne, Dinesh Manocha

PhysGS, developed by researchers at the University of Maryland and Stanford University, integrates Bayesian inference into 3D Gaussian Splatting, leveraging vision-language models, to estimate dense, multi-property physical maps with uncertainty for robotic interaction. The system demonstrates improved accuracy in estimating mass, friction, hardness, and stiffness compared to existing methods, notably reducing Shore hardness error by up to 61.2% and mass estimation error by 22.8%.

Key insights

• Embedding Bayesian inference directly into 3D Gaussian Splatting allows each 3D primitive to be a probabilistic entity, enabling iterative refinement of physical property beliefs from multi-view, confidence-weighted observations.
• Vision-language models can effectively serve as semantic priors, bridging high-level language concepts to low-level physical attributes by providing initial material labels, property estimates, and confidence scores to guide Bayesian updates.
• The explicit decomposition of predictive uncertainty into aleatoric and epistemic components provides a more interpretable and robust measure of confidence, critical for risk-aware robotic decision-making by distinguishing between data noise and a lack of model knowledge.

Summary by alphaXiv.