Reproduce “In Praise of Stubbornness: An Empirical Case for Cognitive-Dissonance Aware Continual Update of Knowledge in LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Simone Clemente, Zied Ben Houidi, Alexis Huet, Dario Rossi, Giulio Franzese, Pietro Michiardi
Researchers from Huawei Technologies and EURECOM found that Large Language Models (LLMs) suffer catastrophic forgetting of unrelated knowledge when updated with contradictory information, losing up to 80% of facts. The study introduces a method to reliably detect such cognitive dissonance in LLMs with over 95% accuracy using internal model features.
Key insights
- Contradictory updates cause catastrophic corruption of unrelated knowledge in LLMs, fundamentally different from the localized impact of non-dissonant updates.
- Selective plasticity can protect existing knowledge during non-dissonant updates but is ineffective against the destructive impact of contradictory information.
- LLMs exhibit detectable internal "signatures of conflict" that allow for highly accurate identification of contradictory information prior to training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.