Reproduce “Hinted Scaling Laws” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Suze van Adrichem, Luke Bailey, Rohith Kuditipudi, Christine Ye, Neil Band, Ahmed Ahmed, John Duchi, Percy Liang and 1 more
The authors develop benchmark hinting to forecast downstream accuracy when smaller models score near zero. They add graded solution help, fit its relationship with model scale, and predict the unhinted score. On Delphi, logit mixing predicted 25.9% GSM8K accuracy for a 10^23-FLOP model within 3 percentage points, using models trained with at most 10^20 FLOPs. This result uses a solution-prompted teacher.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.