Reproduce “Single-Query Person-Centric Bimanual Hand-Object Interaction Detection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jonghyun Kim, Junho Roh, Yubin Yoon, Hyotae Lee, Jongkuk Park, Taehwan Hwang, Jaechul Kim, Jungho Lee
A person-centric transformer detects each individual together with their body pose, left and right hands, contact states, and interaction targets using one shared query. Its relation-based target selection and part-aware deformable attention substantially improve complete hand–object tuple accuracy over direct box regression, reaching 66.2 medium and 32.7 hard accuracy when supplemented with 100DOH data, although performance drops in crowded scenes.
Key insights
- Representing bilateral hand interactions within a shared person query improves association and complete interaction reconstruction rather than merely improving hand-state classification.
- Relation-based target selection outperforms direct box regression by 15.9 points on medium accuracy and 16.3 points on hard accuracy under matched training settings.
- Pose supervision and localized part-aware attention are complementary: they greatly improve hand and joint localization, but introduce a small trade-off in overall detection mAP.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.