Reproduce “Vision Graph Prompting via Semantic Low-Rank Decomposition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zixiang Ai, Zichen Liu, Jiahuan Zhou
This paper introduces Vision Graph Prompting (VGP), a parameter-efficient framework designed for Vision Graph Neural Networks (ViG) that adapts pre-trained models to downstream tasks. VGP leverages semantic low-rank decomposition across three distinct prompt types to capture and enhance global, edge-level, and node-level semantic information in graph-structured visual data. It achieves performance comparable to or exceeding full fine-tuning while reducing trainable parameters by over 94% on average and demonstrates generalizability to traditional graph tasks.
Key insights
- Semantic information in vision graphs primarily resides in low-rank components of the latent feature space, allowing for efficient semantic filtering and amplification.
- Existing visual prompting methods designed for Vision Transformers are not effective for Vision Graph Neural Networks (ViG) due to their disregard for graph topology and semantics.
- Carefully blending intrinsic local features with refined low-rank semantic components is crucial for optimal performance in graph-based visual prompting.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.