Reproduce “Personalized Large Language Models” on OpenResearch

About this paper

Stanisław Woźniak, Bartłomiej Koptyra, Arkadiusz Janz, Przemysław Kazienko, Jan Kocoń

Researchers at Wroclaw Tech developed and evaluated personalized Large Language Models for subjective text perception tasks. They demonstrated that personalized fine-tuning, which integrates user IDs directly into the model's learning process, can yield performance improvements up to 165% on hate speech detection and 64% on emotion recognition compared to non-personalized methods, generally outperforming prompt-based few-shot personalization.

Key insights

• Personalized fine-tuning, which modifies model weights based on user IDs, generally outperforms prompt-based zero-shot and few-shot personalization for subjective NLP tasks, leading to more persistent and effective adaptation.
• The effectiveness of a personalization strategy depends on dataset characteristics; personalized classification (CLS-P) was often better for datasets with many fine-grained labels (e.g., GoEmotions), while personalized language modeling (LM-P) could excel for fewer labels (e.g., Unhealthy Conversations).
• Model architecture and pre-training significantly influence personalization; an encoder-decoder model (Flan-T5 3B) achieved better absolute performance in personalized fine-tuning than a larger decoder-only model (Mistral 7B) for some tasks, and instruction-tuning/alignment is crucial for robust prompt-based personalization.

Summary by alphaXiv.