Reproduce “ParaMETA: Towards Learning Disentangled Paralinguistic Speaking Styles Representations from Speech” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haowei Lou, Hye-young Paik, Wen Hu, Lina Yao
Researchers at the University of New South Wales and CSIRO's Data61 developed ParaMETA, a unified framework that learns disentangled paralinguistic speaking style representations from speech. This framework significantly improves multi-task classification accuracy, enhances the naturalness and expressiveness of synthesized speech by up to 1.0 N-MOS and 0.6 E-MOS respectively, and achieves over 30 times faster inference with 98% fewer parameters compared to large-scale baselines.
Key insights
- Projecting speech into independent, lower-dimensional subspaces for each paralinguistic task effectively mitigates inter-task interference, leading to truly disentangled style representations.
- A 'graded similarity' loss function in a shared META embedding space can effectively group speech samples based on the number of shared style attributes, which is particularly beneficial for disentangling complex, multi-class tasks.
- Maintaining stable class prototypes, dynamically updated during training, provides robust anchor points for both accurate style classification and versatile multimodal style manipulation and control.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.