Reproduce “A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Seliem El-Sayed, Canfer Akbulut, Amanda McCroskery, Geoff Keeling, Zachary Kenton, Zaria Jalan, Nahema Marchal, Arianna Manzini and 12 more
Researchers from Google DeepMind and collaborating institutions developed a conceptual and analytical framework to systematically study and mitigate harms from persuasive generative AI. The work clarifies definitions of AI persuasion, categorizes potential harms with a focus on 'process harms', and maps the underlying psychological mechanisms and contributing model features that enable AI systems to persuade or manipulate, informing targeted mitigation strategies.
Key insights
- Generative AI introduces a 'new risk profile' for persuasion due to its capacity for reciprocal interaction, prolonged engagement, and vast scale of interactions, often with limited human oversight.
- Focusing on mitigating 'process harms' (e.g., degradation of cognitive autonomy) is more universally tractable and effective than solely addressing highly contextual 'outcome harms'.
- AI persuasion operates through distinct psychological mechanisms like trust, anthropomorphism, personalization, deception, manipulative strategies, and alteration of the choice environment, each with specific model features.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.