Reproduce “NeuroSonic: Conditional Flow Matching for EEG-to-Speech Reconstruction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenhao Gao, Yifan Wang, Yijia Ma, Carl Yang, Wen Li, Chenyu You
NeuroSonic introduces a conditional flow-matching framework for reconstructing continuous speech from scalp EEG signals, modeling the problem as a deterministic conditional acoustic transport. The framework achieves superior distributional realism, spectral fidelity, and perceptual quality, demonstrating up to a 31.6% reduction in Fréchet Audio Distance and a 26.3% gain in overall perceptual quality compared to established generative baselines.
Key insights
- Reformulating EEG-to-speech as a deterministic conditional flow matching problem offers greater stability and robustness by directly modeling acoustic trajectory evolution, circumventing the limitations of stochastic multi-step sampling common in diffusion models.
- Velocity-based supervision is crucial for enforcing coherent evolution along the acoustic manifold, leading to improved spectral structure and higher perceived quality compared to direct waveform regression which may only match endpoints.
- A multimodal, time-conditioned Transformer architecture with RMS normalization effectively handles the inherent structural mismatch and nuisance variability between EEG and speech, implicitly attenuating localized motion artifacts and low-SNR perturbations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.