Reproduce “TeHOR: Text-Guided 3D Human and Object Reconstruction with Textures” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hyeongjin Nam, Daniel Sungho Jung, Kyoung Mu Lee
Researchers from Seoul National University introduce TeHOR, a framework for text-guided 3D human and object reconstruction from a single image, which leverages semantic text descriptions and appearance cues to overcome reliance on physical contact. This method achieves state-of-the-art accuracy, particularly in non-contact scenarios, and is the first to jointly reconstruct full 3D textures for both humans and interacting objects.
Key insights
- Semantic guidance from text descriptions is crucial for understanding and reconstructing a wider spectrum of human-object interactions, especially non-contact scenarios.
- Integrating appearance-level supervision through diffusion models provides dense, pixel-level gradients that are more effective than global CLIP embeddings for achieving visually plausible and contextually coherent 3D reconstructions.
- Representing 3D humans and objects as Gaussians, combined with a 2D background, allows for superior modeling of high-fidelity visual appearances and more effective spatial optimization compared to mesh-based methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.