Reproduce “VueBuds: Visual Intelligence with Wireless Earbuds” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Maruchi Kim, Rasya Fawwaz, Zhi Yang Lim, Brinda Moudgalya, Hexi Wang, Yuanhao Zeng, Shyamnath Gollakota
VueBuds is the first camera-integrated wireless earbud system, enabling egocentric visual intelligence and real-time interaction with vision language models (VLMs). The system integrates dual low-power cameras into earbuds, providing robust environmental views and achieving VLM response quality comparable to smart glasses.
Key insights
- Camera hardware, specifically an ultra-low-power module drawing under 5mW, can be successfully integrated into truly wireless earbuds within strict SWaP constraints, with minimal impact on battery life.
- Ear-level camera placement, despite being off-axis, can provide effective egocentric vision by leveraging binocular views, optimized angling, and image processing to minimize blind spots.
- A fully wireless, Bluetooth-based pipeline can support real-time multimodal interaction with VLMs, achieving comparable performance to smart glasses, with potential for sub-2-second end-to-end latency through system optimizations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.