Reproduce “AttriPrompt: Dynamic Prompt Composition Learning for CLIP” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qiqi Zhan, Shiwei Li, Qingjie Liu, Yunhong Wang
Researchers at Beihang University developed AttriPrompt, a framework for CLIP that dynamically composes prompts by leveraging fine-grained visual features from intermediate layers, achieving state-of-the-art generalization and robustness across various image recognition benchmarks.
Key insights
- Dynamically generated prompts, derived from hierarchical visual features, enhance textual representations with fine-grained visual attributes, providing a content-aware adaptation mechanism for VLPMs.
- Applying self-regularization effectively preserves the extensive pre-trained knowledge of VLPMs while adapting to new tasks, preventing overfitting and maintaining broad generalization.
- A dual-stream contrastive learning approach refines the alignment between visual and textual modalities, allowing for a deeper integration of detailed visual cues into the prompt learning process.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.