Reproduce “TIGER-FG: Text-Guided Implicit Fine-Grained Grounding for E-commerce Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinyu Sun, Huangyu Dai, Lingtao Mao, Zexin Zheng, Zihan Liang, Ben Chen, Chenyi Lei, Wenwu Ou
Researchers at Kuaishou Technology developed TIGER-FG, an asymmetric dual-encoder framework that leverages structured item text to implicitly guide fine-grained grounding for e-commerce image-to-multimodal retrieval (IMMR). The model achieves a 34.4 percentage point improvement in Recall@1 on a new challenging benchmark (ECom-RF-IMMR-Mosaic) featuring cluttered, multi-item scenes, outperforming large multimodal embedding models without explicit object detection.
Key insights
- Implicit, text-guided fine-grained grounding, without explicit object detection, can effectively resolve both modality and granularity disparities in e-commerce IMMR, leading to a more robust and efficient system.
- Clutter-aware training, particularly with Mosaic augmentation which synthesizes multi-item candidate scenes, is crucial for developing models that are robust to visual distractors and object co-occurrence.
- Dual distillation objectives (spatial-relational and similarity-distribution) enhance multimodal item representation learning, improving both fine-grained visual focus and overall query-item similarity structure.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.