Reproduce “CLIP-SCGI: Synthesized Caption-Guided Inversion for Person Re-Identification” on OpenResearch

About this paper

Qianru Han, Xinwei He, Zhi Liu, Sannyuya Liu, Ying Zhang, Jinhai Xiang

Researchers from Huazhong Agricultural University and Central China Normal University introduced CLIP-SCGI, a framework that uses synthesized captions to explicitly guide Vision-Language Models for person re-identification. The framework achieved state-of-the-art performance on several benchmarks, including 88.2% mAP and 92.9% Rank-1 accuracy on the challenging MSMT17 dataset.

Key insights

• Explicit semantic guidance, derived from synthesized captions, significantly enhances the learning of discriminative features in CLIP-based Re-ID models compared to implicit prompting methods.
• A caption-guided inversion process can effectively refine image features by re-weighting them based on textual descriptions, translating relevant visual information into a robust pseudo-word token.
• Fusing semantic textual inversion embeddings with global image features through a multi-modal cross-attention mechanism leads to richer and more robust contextual representations for person re-identification.

Summary by alphaXiv.