Reproduce “Streaming keyword spotting on mobile devices” on OpenResearch

About this paper

Oleg Rybakov, Natasha Kononenko, Niranjan Subrahmanya, Mirko Visontai, Stella Laurenzo

Researchers at Google introduced a Keras-based streaming wrapper that automatically converts non-streaming KWS models for efficient on-device inference, demonstrating a 10-20x latency reduction on mobile devices and achieving 98.0% accuracy on Google Speech Commands V2 with a novel Multi-head Attention RNN.

Key insights

• Automatic streaming conversion using the Keras wrapper significantly reduces inference latency on mobile devices by 10-20x, streamlining deployment for real-time applications.
• The proposed Multi-head Attention RNN (MHAtt-RNN) architecture achieves state-of-the-art accuracy on KWS tasks, demonstrating a 10% relative reduction in classification error.
• While effective, specific challenges exist for streaming models, such as potential accuracy degradation when training RNNs in a stateful manner and current limitations for layers with temporal striding/pooling in the automated conversion tool.

Summary by alphaXiv.