Reproduce “Beyond Prompt Adherence: Auditing Attribute-Level Voice Control in Speech Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xianhao Zhou, Jianghao Wu
A new diagnostic framework audits attribute-level voice control in speech generation, finding that current systems frequently introduce unintended collateral changes when modifying a requested voice attribute. The work introduces VoDER-Cal, a training-free candidate selection method, which reduces mean held-out off-target deviation by 20% compared to target-only selection by prioritizing outputs that preserve unrequested attributes.
Key insights
- Prompt adherence and attribute preservation are distinct aspects of controllability; current generative models can achieve desired attribute changes but often introduce significant collateral modifications.
- Different systems implement the same natural language voice descriptors through varied combinations of pitch, timing, energy, and spectral changes, indicating a lack of truly disentangled attribute control.
- Even when a system clearly responds to a target attribute request, substantial and noticeable off-target changes frequently persist, suggesting that clear target response alone is insufficient for localized editing.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.