Reproduce “VCG: A Multimodal Retrieval Framework for E-Commerce Video Feeds under Extreme Cold-Start Conditions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Katya Mirylenka, Egor Malykh, Mahdyar Ravanbakhsh, Michael Gygli, Marco-Andrea Buchmann, Andrew Dzhoha, Svitlana Borzenko, Francesca Catino and 5 more
Researchers at Zalando and TU Wien developed VCG, a multimodal retrieval framework designed to provide relevant e-commerce video recommendations under extreme cold-start conditions using domain-adapted CLIP embeddings. The system achieved a +50.10% lift in video consumption depth (VideoProgress @ 50%) and a 27% increase in average LVLM judge relevance scores during online A/B testing.
Key insights
- Discriminative models like CLIP are more suitable for raw visual retrieval tasks than generative models due to their embedding spaces exhibiting better separability for vector search, despite generative models excelling in attribute prediction.
- An "LVLM-as-a-judge" evaluation framework offers a scalable and robust alternative to human annotation or traditional metrics, especially for assessing semantic relevance in cold-start scenarios with exposure bias.
- Achieving effective cold-start recommendations for dynamic content relies on decoupling content representation learning from user interaction history by using pre-trained, domain-adapted multimodal encoders.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.