Reproduce “Patch-wise Retrieval: A Bag of Practical Techniques for Instance-level Matching” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Wonseok Choi, Sohwi Lim, Nam Hyeon-Woo, Moon Ye-Bin, Dong-Ju Jeong, Jinyoung Hwang, Tae-Hyun Oh

This research presents Patchify, a single-stage framework for instance-level image retrieval that uses structured multi-scale patches encoded by frozen pre-trained visual models. The system achieves high performance, scalability through product quantization, and inherent interpretability, outperforming global descriptors and offering a memory-efficient alternative to complex reranking pipelines.

Key insights

  • Structured, sparse multi-scale patch representations consistently and substantially outperform global descriptors for instance-level matching, especially with challenging object instances.
  • The effectiveness of visual encoders for retrieval is highly dependent on the scale of their pre-training data, with larger datasets yielding better generalization regardless of architecture.
  • Product Quantization effectively compresses patch descriptors for scalable retrieval with minimal performance drop, but the quality of features used for PQ training is critical.
  • The LocScore metric provides valuable diagnostic insights, showing that high retrieval accuracy (mAP) can occur even when matched regions are spatially imprecise.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.