Reproduce “BATON: Aligning Text-to-Audio Model with Human Preference Feedback” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Huan Liao, Haonan Han, Kai Yang, Tianjiao Du, Rui Yang, Zunnan Xu, Qinmei Xu, Jingquan Liu and 2 more
The BATON framework, developed by Tsinghua University and Huawei Technologies, introduces human preference feedback into text-to-audio (TTA) generation to address challenges in aligning generated audio with specific textual details, particularly regarding integrity and temporal relationships. This approach led to improvements in objective alignment metrics, with CLAP scores increasing by up to 6.0% for temporal tasks, and achieved higher human evaluation scores for both quality and faithfulness compared to baseline TTA models.
Key insights
- Human preference feedback, adapted from successful applications in language and image models, effectively improves text-to-audio alignment.
- The BATON framework is designed as a "plug-and-play" solution, suggesting its potential applicability across various existing text-to-audio models.
- Careful balancing of human preference learning with regularization prevents overfitting and maintains overall audio quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.