Reproduce “MMHOI: Modeling Complex 3D Multi-Human Multi-Object Interactions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kaen Kogashi, Anoop Cherian, Meng-Yu Jennifer Kuo
MMHOI introduces a 600,000-frame benchmark for reconstructing and interpreting scenes with multiple people, objects, and simultaneous interactions, with annotations covering 3D geometry, 78 actions, and 14 interacting body-part categories. Its accompanying MMHOI-Net jointly estimates human and object geometry, 6-DoF object pose, actions, and contact regions from a single RGB image, reducing multi-interaction reconstruction error compared with adapted single-HOI baselines.
Key insights
- Interaction semantics can improve 3D reconstruction: action and body-part supervision helps resolve ambiguities that cannot be addressed reliably from image appearance alone.
- The dual-patch object representation progressively improves multi-interaction reconstruction, reducing human/object Chamfer Distance from 30.25/9.35 cm with only the human branch to 21.02/6.47 cm in the full model.
- Jointly modeling all human–object pairs is more effective than independently applying single-interaction methods and aggregating their predictions, particularly in scenes with occlusion and multiple simultaneous interactions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.