Reproduce “ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Wenshuo Li, Xinghao Chen, Han Shu, Yehui Tang, Yunhe Wang

ExCP compresses training checkpoints by encoding changes between nearby weight snapshots and using optimizer momentum to guide joint pruning before quantizing what remains. On Pythia-410M, reported size falls from 4.53G to 0.06G, about 70× compression. The arithmetic mean of six task scores is 43.11 for the original and 42.93 with ExCP, while individual scores vary; evidence is limited to tested evaluations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.