Reproduce “AURA: AUdio-dRiven streaming Avatar” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Nathaniel Cohen, Nicolas Dufour, Amélie Royer, Alasdair Newson, Patrick Pérez

AURA generates audio-driven avatar video in causal chunks, using each chunk’s audio and a fixed-size cache of recent visual context. Distillation turns a bidirectional video generator into a four-step student. On one H100, AURA 1.3B produces 20.7 frames per second, above its 16 Hz playback rate; AURA 5B reaches 9.7 frames per second, below 25 Hz.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.