Reproduce “NextQuill: Causal Preference Modeling for Enhancing LLM Personalization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaoyan Zhao, Juntao You, Yang Zhang, Wenjie Wang, Hong Cheng, Fuli Feng, See-Kiong Ng, Tat-Seng Chua
NextQuill introduces a causal preference modeling framework to enhance Large Language Model personalization by distinguishing and emphasizing preference-driven components in both model predictions and user data. This approach consistently improved personalized text generation across multiple datasets and evaluation metrics, demonstrating improved alignment with user-specific preferences.
Key insights
- Genuine LLM personalization requires isolating and focusing on components causally attributable to user preferences, rather than uniformly treating all predictions or ground-truth tokens.
- Both the generation of model-predicted responses and user-written ground-truth responses are outcomes of underlying causal processes, where user history and characteristics play distinct roles.
- Aligning the model's internal causal preference effects with preference-bearing supervision signals, and emphasizing learning from preference-driven ground-truth tokens, leads to a more precise and effective personalization alignment.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.