Reproduce “MiSTR: Multi-Modal iEEG-to-Speech Synthesis with Transformer-Based Prosody Prediction and Neural Phase Reconstruction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mohammed Salah Al-Radhi, Géza Németh, Branislav Gerazov
MiSTR converts recorded intracranial EEG from spoken speech into synthesized audio. It combines wavelet and prosodic features, a Transformer that predicts a Mel spectrogram, and Iterative Harmonic Phase Reconstruction to reconstruct a waveform. In 10-fold cross-validation with 10 people with epilepsy, MiSTR’s Pearson correlation was 0.91 versus 0.87 for an encoder–decoder baseline, though no ablation identifies the source of improvement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.