Reproduce “DegDiT: Controllable Audio Generation with Dynamic Event Graph Guided Diffusion Transformer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yisu Liu, Chenxing Li, Wanqian Zhang, Wenfu Wang, Meng Yu, Ruibo Fu, Zheng Lin, Weiping Wang and 1 more
DegDiT, a framework for controllable text-to-audio generation, utilizes dynamic event graphs to guide a diffusion transformer, achieving superior temporal precision (F1(Event) up to 0.589) and semantic content alignment. This approach enables fine-grained control over event types, timestamps, and temporal sequences in synthesized audio, yielding high perceptual quality.
Key insights
- Explicitly representing audio events and their temporal interdependencies through dynamic graphs provides a robust mechanism for fine-grained control in text-to-audio generation.
- Curating a high-quality, diverse dataset using a Quality-Balanced Data Selection pipeline, which accounts for event rarity and multiple quality criteria, is essential for training effective and generalizable controllable audio models.
- A multi-dimensional preference optimization approach, Consensus Preference Optimization (CoPO), effectively balances various quality aspects (semantic alignment, temporal accuracy, audio quality) to produce outputs that better match human preferences.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.