Reproduce “Event2Audio: Event-Based Optical Vibration Sensing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mingxuan Cai, Dekel Galor, Amit Pal Singh Kohli, Jacob L. Yates, Laura Waller
Event2Audio recovers sound from vibrating surfaces using active laser illumination and an event camera that tracks speckle motion. Its offline pipeline converts asynchronous brightness changes into optical flow and waveform. In a chip-bag speech test, it reports PESQ 1.60 and STOI 0.81 in 1.5 minutes, versus 1.39, 0.73, and 54 minutes for Sheinin et al.; differing setups limit this comparison.
Key insights
- Combining active laser illumination with event cameras effectively overcomes the sensitivity limitations of passive methods and the complexity/speed issues of traditional active systems.
- Defocusing the imaging setup magnifies subtle surface tilts, making the resulting speckle pattern motion readily detectable by an event camera, which excels at capturing fast dynamic changes.
- Adapting optical flow algorithms to event streams allows for efficient and high-speed reconstruction of vibration signals, providing both near real-time capability and significantly faster offline processing.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.