Reproduce “How iteration order influences convergence and stability in deep learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Benoit Dherin, Benny Avelin, Anders Karlsson, Hanna Mazzawi, Javier Gonzalvo, Michael Munn
The authors introduce backward training, which rebuilds updates from the initial parameters by applying all batches seen so far in reverse order. Under contraction and bounded-displacement assumptions, the backward sequence converges; conditions also relate its distribution to forward training. In CIFAR-10 experiments, curves are smoother across five seeds, but this does not establish lower computation or better test accuracy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.