Reproduce “From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Julia Mendelsohn, Ronan Le Bras, Yejin Choi, Maarten Sap
Researchers at the University of Michigan and the Allen Institute for AI developed a comprehensive taxonomy and glossary of 340 dogwhistles, providing foundational resources for computational analysis of coded rhetoric. Their empirical evaluation demonstrated that large language models show partial recognition of these terms, while widely-used toxicity detection systems consistently fail to identify dogwhistles, allowing covert harmful language to evade moderation.
Key insights
- Dogwhistles consistently evade widely-deployed toxicity detection systems like Perspective API, highlighting a critical vulnerability in current content moderation tools.
- Large language models (LLMs) like GPT-3 show varying capabilities in recognizing dogwhistles, performing better on formal/offline terms and when provided with explicit definitions and cues.
- The historical analysis reveals how dogwhistle usage patterns in political discourse shift over time, reflecting evolving strategies for coded appeals and the dynamic nature of these terms.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.