Reproduce “Dissecting Geometric Constraints in Grokking: Interaction-Dependent Acceleration in Modular Arithmetic” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zachary A. Hampton, Wasiq Khan, Bilal Khan
The authors test how training-recipe components affect grokking in shallow transformers for modular arithmetic. A one-layer factorial varies projection, readout normalization, logit scale, and weight decay; validation compares Sphere+Fourier with a baseline across five partitions. Sphere+Fourier crossed 95% accuracy in 25/25 runs versus 8/25, but the results neither identify projection alone as the cause nor establish transfer beyond these settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.