Reproduce “Weight-Space Programming Doesn't Need Transformers” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hyunjong Ok, Jaeho Lee

The authors test whether current LLMs can write models directly from random initialization without gradient training or running on data. Given three LLMs ten labeled science-question examples, they find sparse tables of written facts outperform the same writers’ fixed Transformer architectures. Claude Opus 5.5’s fifth-generation table model reaches 54.4% test accuracy on ARC; the finding is limited to this setup.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.