Reproduce “Reconstructing Humans and Objects in Interaction using Large Reconstruction Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Agniv Chatterjee, Georgios Pavlakos
MILO, a framework from the University of Texas at Austin, reconstructs 3D human-object interactions from single images by leveraging Large Reconstruction Models (LRMs) to create an initial holistic human-object mesh. This approach, which bypasses the need for ground-truth contact information, achieved the lowest Procrustes-Aligned Chamfer Distance across several benchmarks, including 7.45 cm on InterCap with object templates and 9.36 cm without.
Key insights
- Large Reconstruction Models can effectively provide a robust geometric scaffold for 3D human-object interactions, inherently preserving relative arrangement and proximity cues that simplify subsequent reconstruction steps.
- Reframing the 3D HOI problem as interpreting and refining an LRM-generated holistic mesh allows for competitive accuracy without relying on explicit ground-truth contact annotations.
- Employing a joint human-object reconstruction from the LRM is crucial for capturing accurate interaction cues, outperforming approaches that reconstruct human and object independently or use object-only pipelines.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.