Reproduce “Building a Personalized Dialogue System with Prompt-Tuning” on OpenResearch

About this paper

Tomohito Kasahara, Daisuke Kawahara, Nguyen Tung, Shengzhe Li, Kenta Shinzato, Toshinori Sato

Researchers from Waseda University and LINE Corporation developed a personalized dialogue system using prompt-tuning on pre-trained large language models, generating consistent and natural responses with reduced computational costs. The approach demonstrated superior performance in manual evaluations compared to fine-tuning, particularly for larger models, and effectively used small, persona-specific datasets in both English and Japanese.

Key insights

• Prompt-tuning is an effective parameter-efficient method for imbuing large language models with consistent persona information without extensive fine-tuning.
• The approach significantly reduces computational and memory requirements for adapting large models, making personalized dialogue system development more accessible.
• Personalized dialogue systems can generate natural and consistent persona-based responses even when trained on relatively small persona-specific datasets.

Summary by alphaXiv.