Reproduce “R3-VAE: Reference Vector-Guided Rating Residual Quantization VAE for Generative Recommendation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qiang Wan, Ze Yang, Dawei Yang, Ying Fan, Xin Yan, Siyang Liu, Yicong Liu, Chenwei Zhang and 3 more
R3-VAE, a new framework for generative recommendation developed by the ByteDance Toutiao Recommendation Team, introduces a reference vector-guided rating residual quantization VAE to generate high-quality Semantic Identifiers (SIDs). This approach stabilizes training and offers direct SID quality assessment, leading to superior performance on public and industrial datasets, including a 1.60% increase in StayTime/U and a 15.36% boost in cold-start click volume in online A/B tests.
Key insights
- Introducing a learnable reference vector to project item embeddings before quantization effectively reduces initialization sensitivity and explicitly captures preference-relevant information, enhancing semantic alignment.
- Utilizing a dot product-based rating mechanism for hierarchical quantization provides stable, fine-grained gradient flow, resolving the codebook collapse issue common in STE-based VQ methods.
- Optimizing directly for Semantic Cohesion (intra-cluster consistency) and Preference Discrimination (inter-cluster divergence) during training, which strongly correlate with downstream GR performance, allows for efficient and quantifiable SID quality improvement without extensive A/B testing.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.