Reproduce “Composing People Together: Iterative Pose-Image Generation for Multi-Person Interaction Scenes” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenxuan Peng, Bharath Hariharan, Hadar Averbuch-Elor
Cornell University researchers introduce a method for generating diverse and compositionally accurate multi-person interaction scenes from text by jointly predicting 2D pose visualizations and RGB images. The framework uses an iterative, person-by-person construction process and role-aware positional encoding, leading to superior semantic alignment and diversity compared to existing diffusion models on multi-person benchmarks.
Key insights
- Explicitly integrating person-centric structural priors (pose) directly into diffusion transformers is crucial for overcoming limitations in compositional accuracy and semantic diversity for multi-person interaction scenes.
- Decomposing complex multi-person generation into an iterative, person-by-person construction process simplifies the reasoning and improves the model's ability to ground nuanced interactions.
- Role-aware positional encoding acts as an effective mechanism to bind textual descriptions of individual people to their corresponding visual representations, ensuring consistent semantic grounding in multi-entity contexts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.