Reproduce “Digital Voicing of Silent Speech” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
David Gaddy, Dan Klein
University of California, Berkeley researchers developed a method to convert silent speech muscle impulses into audible synthetic speech, a task termed 'digital voicing of silent speech'. This approach achieved a 3.6% Word Error Rate in a closed-vocabulary task and demonstrated a 20% absolute improvement over baselines in an open-vocabulary setting, showing the ability to generate intelligible speech from silent articulation.
Key insights
- Directly training speech synthesis models on silent EMG, facilitated by transferred audio targets, substantially improves intelligibility compared to models trained solely on vocalized data.
- Physiological differences between silent and vocalized speech can be effectively addressed through techniques like CCA and iterative alignment refinement using predicted audio.
- The performance of silent speech synthesis systems is highly dependent on the quantity and parallelism of the training data, indicating scalability with more resources.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.