Reproduce “Adapting Vision-Language Models for E-commerce Understanding at Scale” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Matteo Nulli, Vladimir Orshulevich, Tala Bazazo, Christian Herold, Michael Kozielski, Marcin Mazur, Szymon Tuzel, Cees G. M. Snoek and 4 more
Researchers at eBay Inc. and the University of Amsterdam developed a systematic recipe for adapting general Vision-Language Models to e-commerce data, utilizing novel, production-grounded benchmarks. Their approach yielded substantial performance improvements on domain-specific tasks while largely preserving broad multimodal capabilities, and enabled a 3.8x inference speedup for certain multi-image compliance tasks.
Key insights
- Domain-specific knowledge embedded within the Large Language Model (LLM) component of VLMs is crucial for achieving superior performance on e-commerce tasks.
- Targeted fine-tuning, coupled with high-quality, precisely annotated labels and optimized image cropping strategies, significantly boosts performance and inference efficiency for complex multi-image tasks.
- Carefully curated single-image instruction tuning can generalize effectively to multi-image e-commerce scenarios, demonstrating transferability of learned capabilities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.