Reproduce “Beyond Masked Sparsity: SNACK Enables Truly Sparse Neural Networks on GPU” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jafar Badour, Maurice van Keulen, Elena Mocanu
SNACK is a PyTorch sparse layer that stores active weights as coordinate-list triples and keeps forward computation, weight gradients, and connection updates sparse. Its custom COO kernel distributes work across nonzero weights and batch columns for small batches. On an RTX 3070, it takes 7.28 ms versus 50.46 ms for Dense+Mask at 95% sparsity and batch 1; it is not universally faster.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.