Reproduce “The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Cedric Flamant, Udaya Ghai, Kanna Shimizu

The Bicameral Model introduces an architecture for two frozen large language models to communicate bidirectionally and continuously through their intermediate hidden states, bypassing token-level text exchanges. This approach enables substantial improvements in tasks requiring external tools, such as achieving 96.5% arithmetic accuracy with a calculator and increasing logic puzzle solving accuracy by 1.7x using a Z3 solver.

Key insights

  • A trainable neural interface can facilitate effective, continuous hidden-state communication between frozen LLMs, enabling emergent coordination without predefined text-based protocols.
  • The learned suppression gates develop selective communication patterns, with forward coupling focusing on task-relevant information and reverse coupling activating primarily when tool outputs become available.
  • A trade-off exists between the capacity of the interface's translation network and generalization, where simpler identity-based coupling can sometimes offer better out-of-distribution performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.