Reproduce “Beyond Accuracy: How Quantization Affects Coding Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Areej Mehboob, Muhammad Ali Shafique, Muhammad Usman Qadeer, Hamza Farooq
The authors study how 4-bit weight-only quantization affects coding agents that execute and revise Python programs. They compare full-precision and quantized models in one execution-feedback setup. For Qwen-7B, AWQ changes successful solutions from 17 one-shot to 0 and from 10 loop-based to 25. Overall accuracy remains similar, but this result does not establish equivalence or generalize beyond the tested setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.