Reproduce “Are We Testing Too Much? A Survey of Benchmark Compression for Efficient Evaluation of Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guoming Ling, Yupei Lin, Zhongzhan Huang, Junxin Li, Junjie Liu, Shanshan Zhong, Hefeng Wu, Liang Lin
The authors survey 119 papers on reducing the work of evaluating large language models. They classify methods by the information they use, the costs they shift or save, and whether they estimate full-evaluation scores, rankings, or decisions. The survey offers a guide for choosing and checking methods, but differing study settings prevent quantitative comparison across them.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.