Reproduce “Don't Think of the White Bear: Ironic Negation in Transformer Models Under Cognitive Load” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Logan Mann, Nayan Saxena, Sarah Tandon, Chenhao Sun, Savar Toteja, Kevin Zhu
Researchers from multiple institutions demonstrated that large language models consistently exhibit 'ironic rebound,' making forbidden concepts more probable under cognitive load, similar to human thought suppression. They introduced ReboundBench, a dataset of 5,000 negation prompts, and identified that specific, sparse attention heads in middle layers of models like Llama-3-8B-Instruct are responsible for amplifying these forbidden tokens.
Key insights
- LLMs exhibit cognitive-like 'ironic rebound' when attempting to suppress thoughts, demonstrating parallels with the human 'white bear' effect.
- The type of cognitive load significantly modulates rebound strength; semantic distractors amplify rebound, while repetition distractors paradoxically aid suppression.
- A model's ability to discriminate between neutral and negative framings of a concept (polarity discrimination) positively correlates with the persistence of ironic rebound.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.