Reproduce “Detecting Multi-Agent Collusion Through Multi-Agent Interpretability” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aaron Rose, Carissa Cullen, Sahar Abdelnabi, Philip Torr, Brandon Gary Kaplowitz, Christian Schroeder de Witt
HELIOS combines per-agent activation signals to test groups of agents for collusion. It trains probes on committee deliberations, then evaluates them zero-shot on structurally different tasks, including coded blackjack signals hidden in ordinary table talk. On four open-weight models, at least one method reaches 0.99–1.00 AUROC on Stego, but results vary by task, with mixed transfer to emergent collusion.
Key insights
- No detector consistently performs best across models and tasks, matching the statistical formulation in which different collusion patterns call for different tests.
- Internal activations can provide evidence that is difficult to obtain from visible dialogue: detectors performed especially well when agents used changing natural-language signals to coordinate.
- Strong ranking performance does not ensure practical detection at strict false-positive thresholds, and transfer from instructed to emergent collusion is limited.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.