Reproduce “S2A2: Audio-Visual Imitation Learning for Manipulation Tasks Using Acoustic Spatial Information” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kaneyoshi Hiratsuka, Benjamin Yen, Ryosuke Kojima
S2A2 equips robot manipulation policies with separate audio representations of sound location and sound identity, alongside images and movement state. Microphone arrays estimate source position in a workspace map, while source-focused spectrograms capture sound identity. In simulation, Diffusion Policy reaches 89.7 ± 2.1% on Localization and Identification, versus 24.7 ± 4.2% for raw audio; real-robot evidence is limited to the tested setting.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.