Reproduce “We Can Hear You with mmWave Radar! An End-to-End Eavesdropping System” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dachao Han, Teng Huang, Han Ding, Cui Zhao, Fei Wang, Ge Wang, Wei Xi
The research introduces "mmSpeech," an end-to-end system that reconstructs intelligible, unconstrained speech from loudspeaker-induced vibrations detected by mmWave radar, operating effectively through walls and without prior knowledge of the speaker's voice. It demonstrates an FWSegSNR of 9.43 dB and a STOI of 0.80 for seen victims, exceeding the performance of prior methods under no-prior-knowledge conditions.
Key insights
- Optimal material properties (high natural frequency, low mass, strong reflectivity) and a precisely chosen radar sampling rate (8000 Hz) are crucial for high-fidelity mmWave vibration capture, effectively preserving high-frequency vocal features.
- A robust GAN-based DNN can effectively transform noisy, narrowband mmWave vibration signals into clear, intelligible wideband speech by denoising spectrograms and restoring missing frequency components.
- Combining synthetic data pre-training with fine-tuning on real-world measurements is an effective strategy to develop generalized mmWave speech reconstruction systems that perform well on unseen speakers and through physical obstructions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.