Reproduce “Zephon: Elastic Determinism for Online, Stateful Foundation Model Data Loading Pipelines” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Maximilian Böther, Josh Wills, Ties Robroek, Sonnet Xu, Paul Burstein, Daniel Zayas, Cody Blakeney, Siddharth Joshi and 9 more
Zephon preserves ordered batches across supported GPU counts with fixed logical lanes and parallel transformations restored to lane order. On resumption, it rebuilds pipeline state from bounded in-flight inputs instead of replaying the full run. Across 8, 16, 32, and 64 GPUs, DCLM Core v1 score spread fell from 0.922 to 0.061 points in a 1B-parameter, 20B-token experiment; this measures topology variation, not average quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.