Reproduce “Soundwich: Video Generation with Layered and Controllable Audio” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhuo Ning, AmirHossein Naghi Razlighi, Sagi Polaczek, Daniel Cohen-Or, Ali Mahdavi-Amiri
Soundwich adapts a frozen joint audio-video generator to produce one shared video and separate audio tracks for requested sources. It uses source-specific guidance, timing controls, and an internal scene-context track to coordinate them. Across 15 scenes, it achieves 93.1% window success and 1.7% off-window activity, versus 15.0% and 69.7% for naive batch expansion using the same model.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.