Reproduce “Benchmarking Benchmark Leakage in Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ruijie Xu, Zengzhi Wang, Run-Ze Fan, Pengfei Liu
Researchers from Shanghai Jiao Tong University and Shanghai Artificial Intelligence Laboratory developed a simple, scalable pipeline to detect benchmark leakage in Large Language Models. This method, which uses perplexity and n-gram accuracy combined with reference benchmark synthesis, revealed widespread training set leakage across numerous open-source LLMs and provided direct evidence of test set contamination in some models. The work proposes a 'Benchmark Transparency Card' to promote clear disclosure of benchmark utilization.
Key insights
- Benchmark leakage, where evaluation datasets are inadvertently or intentionally included in training data, is a prevalent issue across many open-source LLMs.
- Evaluating LLMs accurately requires methods that can detect leakage without access to internal training data or model logits, especially for black-box models.
- Standardized transparency regarding benchmark usage in training, such as through a 'Benchmark Transparency Card,' is essential for fair evaluation and healthy development of the LLM field.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.