Reproduce “A CPU Inside an LLM: Zero-Gradient Compilation of Exact Circuits into Pretrained Weights” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tanmay Sah
The authors graft a deterministic arithmetic circuit into pretrained model weights, enabling signed addition, subtraction, and multiplication without gradient updates, labeled examples, or runtime tool calls. Scale-robust circuit primitives and tokenizer-aligned emission let the language model resume after the answer. On Arith-1200, BF16 grafts reach 99.17% on Granite 4.2 3B and 100.00% on Qwen 3.5 4B, within the suite’s tested formats.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.