Reproduce “General-Purpose User Modeling with Behavioral Logs: A Snapchat Case Study” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qixiang Fang, Zhihan Zhou, Francesco Barbieri, Yozen Liu, Leonardo Neves, Dong Nguyen, Daniel L. Oberski, Maarten W. Bos and 1 more
A General-Purpose Behavioral Log-based User Modeling (G-BLUM) system, developed by a collaboration including Utrecht University, Northwestern University, and Snap Inc., utilizes a customized Transformer model with ALiBi and dual training objectives to create general-purpose user representations from Snapchat behavioral logs. This system effectively predicts reported accounts and user churn with high AUC scores (e.g., ~0.89 and ~0.90 respectively), demonstrating robust applicability in instant messaging platforms.
Key insights
- General-Purpose Behavioral Log-based User Modeling (G-BLUM) is demonstrably effective for instant messaging platforms, extending its applicability beyond e-commerce and search.
- Combining Masked Behavior Prediction and User Contrastive Learning is crucial for training Transformer models to simultaneously encode general behavioral information and capture user-specific distinctions.
- Attention with Linear Biases (ALiBi) provides a practical method for Transformer models to extrapolate effectively to user behavior sequences longer than those encountered during training, enhancing real-world utility.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.