Reproduce “RecGPT: Generative Pre-training for Text-based Recommendation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hoang Ngo, Dat Quoc Nguyen
VinAI Research introduces RecGPT, a series of domain-adapted large language models specifically pre-trained on a vast corpus of recommendation data to leverage textual item and user interaction representations. RecGPT-7B-Instruct demonstrated improved accuracy in rating prediction and competitive to superior performance in sequential recommendation across multiple benchmark datasets.
Key insights
- Continual pre-training on a recommendation-specific textual corpus is essential for adapting general-purpose LLMs to outperform general or ID-based recommendation models.
- Leveraging rich textual representations of item metadata and user interaction histories allows LLMs to more effectively process and generate relevant recommendations compared to symbolic ID-based approaches.
- Rigorous data handling, including preventing data leakage between pre-training, fine-tuning, and evaluation sets, is critical for reliable benchmarking of recommendation systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.