Reproduce “Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montage” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinwei Hu, Xinmiao Huang, Youcheng Sun, Yi Dong, Xiaowei Huang
This study introduces "Generative Montage," a multi-agent framework demonstrating how LLM agents can be manipulated into adopting fabricated hypotheses by strategically sequencing factually correct information. The method achieved an attack success rate of up to 74.4% across various LLM architectures, successfully misleading downstream verification systems through cognitive steering rather than outright falsehoods.
Key insights
- LLMs exhibit a fundamental cognitive vulnerability: their drive for narrative coherence makes them susceptible to 'overthinking,' causing them to construct spurious causal links from strategically presented, individually truthful information.
- Paradoxically, models with stronger reasoning capabilities and those employing Chain-of-Thought prompting showed increased vulnerability, suggesting advanced inference mechanisms can become an attack surface under adversarial cognitive manipulation.
- Cognitive collusion represents a 'cognitive-centric' threat, operating entirely through public channels with factual evidence, thereby bypassing conventional channel-centric detection mechanisms and exposing a stealthy form of manipulation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.