Reproduce “Segment Where You Look: Attention as the Interface Between Language Models and Segmenters” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Eslam Abdelrahman, Hamza Omar Alkaf, Yuheng Li, Siddharth Srinivasan Iyer, Nanxuan Zhao, Eli Shechtman, Krishna Kumar Singh, Mohamed Elhoseiny
AttnSeg improves language-guided image segmentation by passing a trained map of the [SEG] token’s attention over image patches to the decoder. A frozen Sa2VA-4B probe scores 18.2 and 21.5 more gIoU than the hidden-state signal on GS-Eval and ReasonSeg. At 4B, benchmark gains are strongest for object parts; ablations show attention guidance, not interface replacement alone, supplies most improvement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.