Reproduce “The Rise of Darkness: Safety-Utility Trade-Offs in Role-Playing Dialogue Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yihong Tang, Kehai Chen, Xuefeng Bai, Zhengyu Niu, Bo Wang, Jie Liu, Min Zhang
Research from Harbin Institute of Technology, Baidu Inc., and Tianjin University investigates the inherent trade-off between safety and character portrayal utility in LLM-based role-playing, particularly when simulating villainous personas. Their proposed Adaptive Dynamic Multi-Preference (ADMP) method, augmented with Coupling Margin Sampling (CMS), allows agents to dynamically adjust safety and utility preferences, leading to improved content safety without significantly degrading role-playing consistency.
Key insights
- The safety-utility trade-off is a consistent and measurable issue across diverse LLMs in role-playing, with "villain" characters intensifying this conflict.
- "Risk coupling," defined as the semantic alignment between user queries and a villain's harmful attributes, significantly increases the likelihood of unsafe content generation.
- Dynamic preference adjustment, rather than static safety alignment, allows models to better balance character authenticity with content safety in complex interactive scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.