Reproduce “SCoRD: Semantic-Assisted Continual Retriever-Reranker Distillation for LLM-Based Recommendation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Seunghyun Baek, Gyuseok Lee, Seunghan Lee, Wonbin Kweon, Dong Wang, SeongKu Kang
The SCoRD framework enables the efficient and effective co-adaptation of an ID-based retriever and an LLM-based reranker in recommendation systems operating on dynamic data streams. It utilizes a lightweight Semantic Reasoning Assistant for continuous semantic guidance, leading to superior accuracy and a more favorable efficiency trade-off on Amazon Books, Yelp, and Amazon Movies & TV datasets.
Key insights
- A lightweight Semantic Reasoning Assistant can effectively distill and provide reusable, intent-level semantic guidance from LLMs to the retriever, significantly reducing the need for costly, frequent LLM inferences.
- Selectively applying LLM distillation to a small percentage of 'hard' sequences, identified by low retriever confidence, maximizes the impact and efficiency of the LLM's valuable supervision.
- Incorporating intent drift-aware negative sampling and collaborative pseudo-labels enables the retriever to adapt more effectively to evolving user preferences and new users without relying on constant LLM updates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.