Reproduce “OneHOI: Unifying Human-Object Interaction Generation and Editing” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiun Tian Hoe, Weipeng Hu, Xudong Jiang, Yap-Peng Tan, Chee Seng Chan

OneHOI, a unified diffusion transformer framework, integrates human-object interaction (HOI) generation and editing within a single model. This framework achieves state-of-the-art performance in layout-free HOI editing, showing a 10-16% improvement in Editability-Identity and HOI Editability scores, and also advances HOI generation quality while enabling novel multi-HOI editing capabilities.

Key insights

  • Unifying HOI generation and editing allows for a synergistic effect, where broad interaction semantics learned during generation provide deeper structural HOI knowledge for more plausible edits.
  • Explicitly embedding interaction structure into a DiT through components like Action Grounding (union of subject/object regions) and verb-mediated attention significantly improves HOI correctness and coherence.
  • Specialized identity cues (HOI Encoder) and instance-specific positional indexing (HOI RoPE) are crucial for preventing role confusion and feature leakage in complex multi-HOI scenes.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.