Reproduce “Continual Low-Rank Adapters for LLM-based Generative Recommender Systems” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hyunsik Yoo, Ting-Wei Li, SeongKu Kang, Zhining Liu, Charlie Xu, Qilin Qi, Hanghang Tong

PESO updates a single low-rank adapter for next-item recommendation, keeping pretrained language-model weights frozen and penalizing changes from the previous adapter while fitting new interactions. Across three Amazon product categories, it leads or ties every reported result cell: Instruments NDCG@5 is 0.0138 versus 0.0130, a reported 6.15% gain, while Books H@5 ties. Its directional explanation is theoretical and assumption-dependent.

Key insights

  • Continual learning in recommender systems uniquely requires not only preserving long-term preferences (stability) but also actively overwriting or de-emphasizing outdated preferences (plasticity), differing from general continual learning objectives.
  • A theoretically grounded analysis shows that the optimal LoRA state with the proximal regularizer is a generalized eigen-interpolation between the current data optimum and the previous state, with weights determined by data support.
  • The softmax-KL divergence acts as an effective proximal regularizer by implicitly providing a module-wise, previous-state-aware metric that fosters stability while allowing for flexible adaptation in the LoRA parameter space.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.