Reproduce “TF-MLPNet: Tiny Real-Time Neural Speech Separation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Malek Itani, Tuochao Chen, Shyamnath Gollakota
TF-MLPNet is a causal speech-separation architecture designed for hearing aids and other battery-powered hearables. By replacing frequency-domain recurrence with parallel MLP mixing and implementing temporal recurrence through convolutions, it achieves real-time inference on a GAP9 processor with a 483 kB model, 5.6 ms runtime, and 80.1 mW power consumption, while retaining 13.52 dB SI-SDR.
Key insights
- Hardware execution patterns are treated as part of the model design: parallel fully connected layers and 1D convolutions can be more suitable for tiny accelerators than architectures with stronger GPU-oriented primitives.
- Uniform INT8 quantization reduces SI-SDR substantially, whereas selectively assigning higher precision to input/output paths, recurrent states, and selected MLP activations provides a better quality-latency balance.
- The MLP-based model benefits more from additional speaker diversity: with the full training set it reaches 14.12 dB SI-SDR in floating point and generalizes to VCTK with 12.63 dB SI-SDR.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.