Reproduce “Dory1.0-4B: A Looped Normalized-GPT” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Maxim Fishman, Simeng Sun, Shantanu Acharya, Faisal Ladhak, Brian Chmiel, Santiago Akle Serano, Ron Banner, Boris Ginsburg and 1 more
Dory1.0-4B tests whether repeating a shared internal block can improve reasoning without producing longer text. It applies the same recurrent block multiple times to its hidden state, adding inference computation between generated tokens. Across an equal-weight average of 12 benchmark scores, results rise from 35.17 at one loop to 40.60 at four, though gains vary by task.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.