Reproduce “MMGRid: Navigating Temporal-aware and Cross-domain Generative Recommendation via Model Merging” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianjun Wei, Enneng Yang, Yingpeng Du, Huizhong Guo, Jie Zhang, Zhu Sun
The MMGRid framework systematically investigates applying model merging to generative recommendation models under real-world contextual shifts, specifically addressing temporal user behavior evolution and heterogeneous application domains. This work identifies challenges related to parameter conflicts and recency bias, proposing methods for effective parameter integration and demonstrating improved performance through weighted merging informed by domain characteristics.
Key insights
- Cross-domain merging for Semantic Indexing (SemID) and Semantic Embedding (SemEmb) GR paradigms suffers from performance degradation due to large, task-aware parameter shifts and architectural conflicts (e.g., out-of-distribution tokens or different learning objectives).
- Simply using the most recent checkpoint in temporal merging introduces recency bias, leading to suboptimal performance; an effective balance between historical and emerging user preferences requires weighted contextual merging, with optimal weights varying significantly across domains and user groups.
- A strong correlation exists between domain characteristics (like average interaction time gap) and optimal temporal merging weights for non-active users, offering a data-driven approach to predict effective weighting without relying on labeled test data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.