Reproduce “Beyond Co-Generation: Rethinking Jointness in Joint Audio-Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jihoo Jung, Ji-Hoon Kim, Suyeon Lee, Gia Thinh Pham, Joon Son Chung
The authors introduce JointREPA to improve semantic coordination between generated audio and video when prompts leave shared details unspecified. It aligns each modality’s features with pretrained representations of both modalities during post-training. On UNSAIDBench, LTX-2.3 (22B) raises AV-IB from 0.244 to 0.302 and AVHScore from 0.236 to 0.288; metrics improve for two tested backbones across three benchmarks, measuring semantic coherence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.