Reproduce “AutoCompress: Automated Hardware-Guided Model Compression” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sheldon Liang, Ali Taghibakhshi, Saurav Muralidharan, Sharath Turuvekere Sreenivas, Ashwath Aithal, Pavlo Molchanov, Nima Tajbakhsh

AutoCompress searches for model architectures against deployment hardware and workload constraints, using profiled serving costs, per-layer structure choices, and distillation. On an RTX 5080 local-coding contract, it meets the 11.5-second latency and 13.96-GiB memory limits, scoring an eight-benchmark average of 68.90, versus 66.55 for Minitron++ with matched 10B recovery budgets. Minitron++ scores higher on three benchmarks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.