Reproduce “One4all User Representation for Recommender Systems in E-commerce” on OpenResearch

About this paper

Kyuyong Shin, Hanock Kwak, Kyung-Min Kim, Minkyu Kim, Young-Jin Park, Jisu Jeong, Seungjae Jung

NAVER Corporation developed ShopperBERT, a model that learns general-purpose user representations by adapting the BERT architecture to large-scale e-commerce purchase histories. These pretrained user embeddings generally outperform task-specific models across six diverse downstream e-commerce tasks, showing notable gains in cold-start scenarios and demonstrating performance improvements directly tied to the scale of pretraining data.

Key insights

• Large-scale pretraining on user purchase histories enables the learning of robust, general user representations that are transferable across diverse e-commerce tasks.
• The performance of learned representations consistently improves as the size of the pretraining dataset increases, demonstrating the importance of data scale in this domain.
• ShopperBERT's embeddings implicitly capture meaningful user characteristics, such as demographics and behavioral similarities, without explicit supervision.

Summary by alphaXiv.