Reproduce “DiveUp: Learning Feature Upsampling from Diverse Vision Foundation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaoqiong Liu, Heng Fan
DiveUp presents a feature upsampling framework for Vision Foundation Models (VFMs) that employs multi-VFM relational guidance to construct a robust Consensus Center-of-Mass Field, leading to improved spatial accuracy and enhanced zero-shot generalization. The method consistently achieves state-of-the-art performance across semantic segmentation and depth estimation benchmarks while maintaining computational efficiency.
Key insights
- Leveraging structural consensus from diverse VFMs provides more robust and accurate geometric guidance for upsampling than relying solely on self-reconstruction from a single source model.
- A universal relational feature representation, such as local Center-of-Mass fields, can effectively bridge disparate feature spaces across various VFMs, enabling seamless cross-model interaction and knowledge transfer.
- Dynamically assessing and fusing contributions from multiple VFMs based on their spatial reliability (e.g., low entropy, low spikiness) is crucial for mitigating artifacts and capitalising on the strengths of individual 'expert' models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.