Reproduce “Can Language Models Discover Scaling Laws?” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haowei Lin, Haotian Ye, Wenzheng Feng, Quzhe Huang, Yujun Li, Hubert Lim, Zhengrui Li, Xiangyu Wang and 3 more

An AI agent designed to discover scaling laws autonomously generated mathematical models that surpass human-derived laws in accuracy and robustness on a new, comprehensive benchmark. This system, called SLDAgent, consistently improved prediction performance for foundation model scaling behaviors and offered practical utility in hyperparameter optimization and model selection.

Key insights

  • AI agents can achieve 'superhuman' performance in open-ended scientific discovery, consistently outperforming human experts and other AI agents in formulating robust, extrapolative scaling laws.
  • The co-evolution of both the mathematical expression and its optimization routine is crucial for achieving superior performance, as tailoring the fitting method to the law enhances accuracy and robustness.
  • SLDAgent often discovers conceptually more principled and dimensionally consistent laws compared to human-derived ones, offering more stable asymptotic behavior and better interpretability.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.