Reproduce “Continuous Thought Machines” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Luke Darlow, Ciaran Regan, Sebastian Risi, Jeffrey Seely, Llion Jones
The Continuous Thought Machine tests whether neuron activity histories and their relationships can guide computation. Each neuron processes its history, while synchronization between histories forms representations for attention and prediction. On mazes without positional embeddings, it generalizes from 39×39 mazes to 99×99 mazes and longer routes; ablations support the combined design in this task, not planning or efficiency claims.
Key insights
- Reintroducing biologically inspired temporal dynamics at the neuron level and treating synchronization as a core computational primitive can unlock capabilities like adaptive compute and complex sequential reasoning in AI systems.
- By allowing individual neurons to have unique temporal processing models, the CTM fosters the emergence of rich, interpretable internal strategies for problem-solving.
- Designing an architecture around an internal, self-generated 'thought sequence' enables iterative refinement of representations and the formation of internal world models, even for static inputs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.