Reproduce “Probing the Mid-level Vision Capabilities of Self-Supervised Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xuweiyi Chen, Markus Marks, Zezhou Cheng
Researchers from the University of Virginia and California Institute of Technology evaluated 22 self-supervised learning models on a new benchmark suite of 8 mid-level vision tasks. They found a weak correlation between high-level vision performance and 3D mid-level understanding, with generative models like iBOT and MaskFeat generally excelling in mid-level capabilities while MAE underperformed.
Key insights
- Strong performance on high-level vision tasks does not automatically imply robust mid-level vision capabilities, particularly for 3D geometric understanding.
- Generative pretraining objectives, such as masked image modeling (e.g., MaskFeat with HOG features), can foster representations richer in fine-grained spatial and geometric information than some contrastive methods.
- Vision Transformers generally learn more comprehensive mid-level features than ResNet architectures, and increased model capacity consistently improves mid-level vision capabilities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.