Reproduce “The rising costs of training frontier AI models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ben Cottier, Robi Rahman, Loredana Fattorini, Nestor Maslej, Tamay Besiroglu, David Owen
Researchers from Epoch AI and Stanford University quantified the escalating costs of training frontier AI models, finding a 2.4x annual increase in amortized training costs since 2016 and estimating a single training run could exceed $1 billion by early 2027. The analysis details expenses across hardware, energy, and R&D staff, showing staff costs can comprise up to 49% of total development expenditures for models like Gemini Ultra.
Key insights
- R&D staff costs are a substantial component of total model development expenses, ranging from 29% to 49% for the deeply analyzed frontier models when including equity compensation.
- Hardware acquisition costs are one to two orders of magnitude higher than amortized training costs, creating a significant upfront capital barrier for AI development.
- The rapid increase in training costs is poised to concentrate frontier AI development among a few well-funded entities, potentially impacting research diversity and market structure.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.