Reproduce “On-Device Large Language Models for Sequential Recommendation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xin Xia, Hongzhi Yin, Shane Culpepper

Researchers developed OD-LLM, a task-adaptive compression framework allowing large language models to perform sequential recommendation directly on devices. The framework reduces model size by approximately 50% while maintaining or improving recommendation quality and achieving up to 3.5x faster inference compared to other leading compression methods.

Key insights

  • Effective on-device deployment of large language models for sequential recommendation is achievable through a specialized task-adaptive compression framework.
  • Token covariance normalization is critical for preparing LLM weight matrices for effective SVD-based compression, ensuring numerical stability and preservation of key features.
  • A progressive, layer-wise refinement strategy significantly mitigates performance degradation during aggressive compression, leading to better accuracy-efficiency trade-offs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.