Reproduce “Test-Time Optimization for Domain Adaptive Open Vocabulary Segmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ulindu De Silva, Didula Samaraweera, Sasini Wanigathunga, Kavindu Kariyawasam, Kanchana Ranasinghe, Muzammal Naseer, Ranga Rodrigo
A framework called Seg-TTO advances zero-shot open-vocabulary semantic segmentation by adapting visual and textual features at test time for specialized domains, achieving an average +2.6% mIoU improvement for the CAT-Seg-L model on the challenging MESS benchmark and up to a 27% mIoU increase on specific datasets.
Key insights
- Segmentation's dense prediction nature requires specialized TTO strategies, as directly applying classification-oriented TTO or naive LLM prompt injections can lead to performance degradation.
- A self-supervised objective, combined with carefully designed visual feature aggregation, is effective for maintaining spatial coherence and improving accuracy during test-time adaptation in semantic segmentation.
- LLMs can significantly enrich textual understanding for domain-specific categories when prompted to generate attributes that *distinguish* concepts, rather than just provide general descriptions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.