Reproduce “ArchSpace: Recording, Reviewing, and Referencing LLM Architecture Research” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jie Wang, Tong Zhu, JiaHao Kuang, Zhe Xu, Yunkai Chai, Dehong Zheng, Yu Cheng, Tao Ji and 3 more
ArchSpace proposes a workflow for recording and testing language-model architecture ideas against baselines in staged experiments. Its coupled scaling law models loss across model size and training tokens, with a term for learning-rate and batch-size choices, and predicts loss on four frozen-law extrapolation runs with 0.32% MAPE. This supports screening under tested conditions, not skipping tests or proving architecture gains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.