Reproduce “End-to-end Audiovisual Speech Activity Detection with Bimodal Recurrent Neural Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fei Tao, Carlos Busso
The authors present end-to-end audiovisual speech activity detection, classifying speech and non-speech frames by learning from sound and mouth images. Separate recurrent networks model each modality over time, and a third models their timing together. In the noisy tablet-sensor test, the system achieves 92.7% F1, 1.2 points above the audio-only baseline, a result limited to that corpus and setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.