Reproduce “BatVision: Learning to See 3D Spatial Layout with Two Ears” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jesper Haahr Christensen, Sascha Hornauer, Stella Yu
BatVision predicts indoor depth maps from binaural echoes: it emits a chirp, records returns with two microphones, and learns from stereo-camera depth labels used only during training. On a held-out floor of one office building, its 16×16 generator-only raw-waveform model achieves 0.0838 L1 loss, below the 0.1058 mean-depth baseline; this shows improvement over that baseline, not navigation performance.
Key insights
- It is possible to reconstruct detailed 3D spatial layouts (depth maps) and plausible grayscale images of an environment from only two 1D binaural audio signals (echoes of emitted chirps).
- An adversarial training approach (GAN) significantly improves the visual quality and detail of the generated depth maps and grayscale images, even if L1 error might slightly increase.
- Using audible frequencies and low-cost, off-the-shelf components allows for a simple, reproducible, and deployable acoustic perception system.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.