Reproduce “Pi-HOC: Pairwise 3D Human-Object Contact Estimation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sravan Chittupalli, Ayush Jain, Dong Huang
Pi-HOC introduces a framework for estimating dense 3D semantic contact between humans and objects directly from an RGB image, adept at handling multiple human-object pairs concurrently. It improves contact prediction F1 score by 11.4% on MMHOI and is up to 88.5 times faster than prior methods, enabling efficient and instance-aware interaction analysis.
Key insights
- Explicitly modeling and refining instance-specific human-object tokens within an InteractionFormer, initialized with DINOv2-L weights, is crucial for disambiguating contacts in complex multi-instance scenes.
- The SAM-based decoder effectively leverages robust image features to accurately localize fine-grained, per-vertex contact on 3D human meshes, significantly outperforming simpler MLP-based decoders.
- A contact-presence classifier allows for efficient early pruning of non-interacting pairs, substantially improving inference speed and scalability in scenes with many potential human-object combinations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.