Reproduce “OW-DETR: Open-world Detection Transformer” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Akshita Gupta, Sanath Narayan, K J Joseph, Salman Khan, Fahad Shahbaz Khan, Mubarak Shah

OW-DETR adapts a transformer detector to identify unfamiliar objects and learn selected new categories. It uses image-feature attention to select unmatched object queries as pseudo-unknowns, then trains an unknown-class branch and a separate foreground-objectness branch. On MS-COCO Task 1, unknown recall is 7.5%, versus 4.9% for ORE without EBUI; this measures retrieval of annotated unknown instances, not all unknown objects.

Key insights

  • Transformer architectures, particularly Deformable DETR, with their multi-scale context modeling and reduced inductive biases, are well-suited for generalizing to diverse and unseen object categories in open-world settings.
  • Attention maps from the backbone can be effectively utilized for bottom-up pseudo-labeling of unknown objects, providing a less biased alternative to RPN-based proposals.
  • Separating class-specific novelty classification from general foreground objectness improves the model's ability to detect unknown objects and transfer knowledge from known to unknown categories.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.