Reproduce “Hard Patches Mining for Masked Image Modeling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haochen Wang, Kaiyou Song, Junsong Fan, Yuxi Wang, Jin Xie, Zhaoxiang Zhang
This paper presents Hard Patches Mining (HPM), a new Masked Image Modeling paradigm where the pre-training model learns to identify and mask its own 'hard' patches. By dynamically focusing on discriminative regions, HPM achieves stronger visual representations, outperforming MAE and other baselines on classification, detection, and segmentation tasks with increased training efficiency.
Key insights
- Reconstruction loss naturally serves as a reliable metric for a patch's difficulty and discriminativeness within an image.
- Learning the relative difficulty among patches using Binary Cross-Entropy is more robust and effective than regressing absolute loss values.
- Gradually increasing the proportion of 'hard' patches selected for masking during pre-training (easy-to-hard strategy) is crucial for optimal performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.