Reproduce “End-to-End Spatial-Temporal Transformer for Real-time 4D HOI Reconstruction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoyu Zhang, Wei Zhai, Yuhang Yang, Yang Cao, Zheng-Jun Zha
Researchers from the University of Science and Technology of China developed THO, an end-to-end Spatial-Temporal Transformer that reconstructs real-time 4D human-object interactions from monocular video. This framework achieves improved accuracy and temporal smoothness, operating at 31.5 frames per second.
Key insights
- Leveraging human-guided contact priors (semantic correspondence and spatial proximity) allows the model to infer occluded object features and refine object representations.
- Explicitly modeling cross-frame kinematic correlations between human joints and the object is essential for achieving long-term physical coherence and smooth motion.
- An end-to-end, feed-forward transformer architecture can simultaneously achieve high reconstruction accuracy, temporal consistency, and real-time inference speed for complex 4D HOI tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.