Reproduce “UniECS: Unified Multimodal E-Commerce Search Framework with Gated Cross-modal Fusion” on OpenResearch

About this paper

Zihan Liang, Yufei Ma, ZhiPeng Qian, Huangyu Dai, Zihan Wang, Ben Chen, Chenyi Lei, Yuqing Ding and 1 more

Kuaishou Technology developed UniECS, a unified multimodal e-commerce search framework, featuring a gated cross-modal fusion encoder and a comprehensive loss design. It achieved state-of-the-art performance on a new M-BEER benchmark, outperforming larger models while using significantly fewer parameters, and demonstrated substantial improvements in business metrics like a 27.31% increase in same-style coverage rate and an 8.33% increase in total revenue upon real-world deployment.

Key insights

• Adaptive gating mechanisms in multimodal fusion are critical for handling varied input modalities and achieving robust performance, especially when modalities are missing or noisy.
• A meticulously designed, multi-faceted loss function and adaptive weighting scheme are essential for training models to learn unified, discriminative, and semantically aligned representations across modalities.
• State-of-the-art unified multimodal retrieval can be achieved with significantly higher computational efficiency (0.2 billion parameters) compared to larger MLLM-based models, making industrial deployment feasible.

Summary by alphaXiv.