Reproduce “Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hyeongjin Nam, Daniel Sungho Jung, Gyeongsik Moon, Kyoung Mu Lee
Researchers at Seoul National University introduce CONTHO, an end-to-end framework for jointly reconstructing 3D human and object meshes from a single image by explicitly leveraging estimated human-object contact information. The method achieves state-of-the-art accuracy in both contact estimation and 3D reconstruction, effectively mitigating undesired human-object correlations for more plausible interaction representations.
Key insights
- Explicitly integrating 3D-guided human-object contact estimation within a joint reconstruction framework significantly improves both contact accuracy and 3D geometric plausibility.
- Using a 'contact-based masking' strategy in a refinement transformer is effective for addressing and mitigating 'undesired human-object correlations,' leading to more realistic and contextually aware 3D interactions.
- Combining 2D visual context with 3D positional information as '3D vertex features' provides a robust input for Transformer-based networks to estimate accurate human-object contact.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.