Reproduce “Gradient-Attention Guided Dual-Masking Synergetic Framework for Robust Text-based Person Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianlu Zheng, Yifan Zhang, Xiang An, Ziyong Feng, Kaicheng Yang, Qichuan Ding
A new framework called GA-DMS, combined with the large-scale WebPerson dataset, improves text-based person retrieval by overcoming data scarcity and enhancing fine-grained cross-modal alignment, achieving new state-of-art performance with up to 10.10% Rank-1 accuracy gain on RSTPReid.
Key insights
- Gradient-Attention Similarity Score (GASS) effectively utilizes fine-grained information from intermediate transformer layers' gradients and attention maps to robustly identify and weight the contribution of individual textual tokens to image-text alignment.
- A synergistic dual-masking approach, simultaneously addressing noisy token suppression and informative token prediction, significantly enhances both cross-modal alignment and the learning of discriminative fine-grained semantic features.
- High-quality, large-scale synthetic datasets, when generated through a noise-resistant pipeline leveraging MLLMs and template-based prompting, can substantially improve the generalizability and performance of vision-language models for specialized tasks like person retrieval.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.