Reproduce “In-Hand 3D Object Reconstruction from a Monocular RGB Video” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shijian Jiang, Qi Ye, Rengan Xie, Yuchi Huo, Xiang Li, Yang Zhou, Jiming Chen
Researchers from Zhejiang University and OPPO US Research Center developed a method for reconstructing complete 3D geometry of objects manipulated by a hand from a monocular RGB video. Their approach integrates 2D amodal masks and 3D physical contact constraints into an implicit neural rendering framework, resulting in 52% higher geometric accuracy and significantly reduced object-hand interpenetration compared to previous techniques.
Key insights
- Incorporating non-observational priors, such as 2D amodal completion and 3D physical contact constraints, is critical for achieving complete and accurate 3D reconstruction of objects heavily occluded by a hand.
- Refining 2D amodal masks for multi-view consistency through joint 3D optimization effectively guides implicit neural representations to reconstruct unobserved surfaces.
- Physical constraints, including non-penetration and a nuanced attraction loss, are crucial for accurately inferring local surface geometry and ensuring physical plausibility in occluded hand-object contact zones.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.