Reproduce “The Functional Structure of Post-Compression Recovery in Low-Rank LLMs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zishan Shao, Liang Tian, Georgiy Zemlevskiy, Kangning Cui, Lixun Zhang, Yixiao Wang, Ting Jiang, Jinhee Kim and 5 more

The authors study how compressed language models change during recovery. They compare loss responses to matched, small channel-rescaling probes at corresponding outputs of compressed models and dense references, calling the difference recovery pressure. Across tested settings, pressure redistributes nonuniformly; in one Qwen example, a model ranked third finishes first, showing that initial rankings can change under recovery.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.