Reproduce “Weakly Supervised Open-Vocabulary Object Detection” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jianghang Lin, Yunhang Shen, Bingquan Wang, Shaohui Lin, Ke Li, Liujuan Cao

A framework for weakly supervised open-vocabulary object detection, WSOVOD, allows detection of both known and unseen object categories using only image-level labels. It integrates a data-aware feature extractor, SAM-guided proposal generation, and a multiple-instance learning network, achieving state-of-the-art results in traditional WSOD and comparable performance to fully-supervised methods on novel open-vocabulary classes.

Key insights

  • Integrating the Segment Anything Model (SAM) as a high-level semantic prior significantly improves the quality and recall of object proposals, even when training under weak supervision.
  • Dynamically adapting feature representations through a Data-Aware Feature Extractor effectively addresses dataset distribution shifts, enabling robust cross-dataset generalization without explicit domain-specific encoders.
  • A multiple-instance learning network, explicitly synchronizing visual proposals with pre-trained textual concept embeddings, facilitates robust open-vocabulary classification and precise object localization using only image-level supervision.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.