Reproduce “Efficient Neural Network Training via Forward and Backward Propagation Sparsification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiao Zhou, Weizhong Zhang, Zonghao Chen, Shizhe Diao, Tong Zhang
The method trains channel-sparse networks with sparse forward and backward computation for weight updates, while updating channel-selection probabilities with two sparse forward passes and no backward pass. Its variance-reduced policy-gradient estimator has bounded variance under stated assumptions. On CIFAR-10 with VGG-16, it matches GrowEfficient at 92.5% validation accuracy, with FLOPs-based train-cost savings of 8.69× versus 1.22×, not elapsed-time speedups.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.