Reproduce “FS-DETR: Few-Shot DEtection TRansformer with prompting and without re-training” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Adrian Bulat, Ricardo Guerrero, Brais Martinez, Georgios Tzimiropoulos

FS-DETR introduces a Few-Shot DEtection TRansformer leveraging visual prompting and pseudo-class embeddings to enable accurate object detection of novel classes without model re-training. This approach achieves state-of-the-art performance for re-training-free methods and rivals re-training based systems, particularly in extreme low-shot conditions, allowing for on-the-fly detection of multiple novel objects.

Key insights

  • Visual prompting combined with pseudo-class embeddings provides an effective mechanism for conditioning a DETR-based model, enabling re-training-free few-shot object detection across multiple novel classes concurrently.
  • The integration of a Multi-Head Cross-Attention (MHCA) layer in the encoder and Type-Specific MLPs (TS-MLP) in the decoder are critical for early filtering of relevant image regions and semantically differentiating template and query features.
  • Unsupervised pre-training significantly boosts FS-DETR's performance, proving essential for preventing overfitting to base classes and improving generalization ability in transformer-based few-shot settings.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.