Reproduce “Annotation-free Audio-Visual Segmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinxiang Liu, Yu Wang, Chen Ju, Chaofan Ma, Ya Zhang, Weidi Xie
SAMA-AVS uses category-matched image masks and labeled audio to train audio-visual segmentation, injecting audio features through small adapters in SAM’s fixed image encoder. Its synthetic dataset contains 62,609 instances across 46 categories, with minimal manual verification. On AVSBench S4, it reaches 81.53 mIoU versus AP-SAM’s 69.61 without synthetic pretraining; this comparison favors the full design but does not isolate encoder fusion alone.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.