Reproduce “Harnessing Intrinsic Subject-Aware Attention for Controllable Multi-Subject Video Generation” on OpenResearch

About this paper

Niange Yu, Ye Tian, Biaolong Chen, Miao Lu, Aixi Zhang, Hao Jiang, Yunhai Tong, Pipei Huang

DIAL uses attention maps within a Diffusion Transformer to locate reference-grounded regions and guide subject-to-video generation. Late in denoising, it applies user-set attention bias; early on, it ranks seeded trajectories by map-mask alignment to create preferences without human labels. On Kaleido, FaceSim rises from 43.80% to 57.46% as shared γ increases from 2 to 8, while NaturalScore declines, limiting generalization.

Key insights

• Attention grounding quality varies across DiT layers and denoising time: maps are diffuse early but become reliable during roughly the final 80% of denoising.
• Localized attention control improves the fidelity–naturalness tradeoff over uniform guidance; on Kaleido, ISGM guidance reaches FaceSim 60.64% with NaturalScore 75.23, compared with 59.21% and 69.03% for the uniform baseline.
• The two phases address complementary failure modes: inference guidance refines subject identity locally, while preference optimization anchors subject presence and consistency during the high-noise stage.

Summary by alphaXiv.