Reproduce “Continuous-Time Acoustic Modelling with Neural Controlled Differential Equations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mattias Cross, Minghui Zhao, Anton Ragni
The authors introduce a text-to-speech method that uses phone durations to produce changing acoustic representations. A neural controlled differential equation evolves a hidden state along a phone-and-time path whose samples feed a standard decoder. In an ESD emotion-intensity test, a one-layer model at h=1.0 reaches macro Spearman correlation 0.53, versus 0.08 for the finetuned baseline; results vary by emotion.
Key insights
- Duration influences the values of the acoustic representation throughout a phone, rather than only determining how many times a static embedding is copied.
- The CDE and matched LSTM models behave differently despite receiving the same duration-augmented inputs, suggesting that the continuous-time formulation contributes more than generic sequential modelling alone.
- Temporal resolution and CDE depth have non-monotonic effects: finer solver steps or additional layers do not consistently improve objective metrics or emotion tracking.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.