Reproduce “Distributed and Decentralised Training: Technical Governance Challenges in a Shifting AI Landscape” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jakub Kryś, Yashvardhan Sharma, Janet Egan
An analysis of the shifting landscape of AI training, informed by recent low-communication algorithms, distinguishes between distributed and decentralized paradigms to highlight emerging governance challenges. The work identifies increased risks of capability proliferation and compute structuring due to these shifts, while also acknowledging potential benefits for democratizing AI access.
Key insights
- Low-communication training algorithms fundamentally alter the physical constraints of AI training, making it feasible to train large models over typical internet speeds and disperse computational efforts.
- Decentralized training, while potentially democratizing AI access, increases the risks of AI capability proliferation, erodes the "shutdownability" of advanced AI systems, and facilitates "compute structuring" to evade oversight.
- Despite the shifts in training paradigms, the fundamental importance of compute quantity and quality (specifically GPU memory and intra-node networking) remains critical for developing frontier AI systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.