Reproduce “Evolution of Benchmark: Black-Box Optimization Benchmark Design through Large Language Model” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chen Wang, Sijie Ma, Zeyuan Ma, Yue-Jiao Gong

Researchers at South China University of Technology developed Evolution of Benchmark (EoB), an LLM-driven framework that automates the design of Black-Box Optimization (BBO) benchmark functions. This method generates diverse and challenging benchmarks by co-evolving function programs and their landscape characteristics, which enhances the generalization of learning-assisted optimizers and provides efficient synthetic proxies for expensive real-world problems.

Key insights

  • Large Language Models (LLMs) can be leveraged for program evolution to automatically generate diverse and objective Black-Box Optimization (BBO) benchmark functions.
  • Framing benchmark design as a bi-objective optimization problem, maximizing Landscape Similarity Indicator (LSI) and Algorithm Distinguishing Capability (ADC), effectively guides automated function creation.
  • An LLM-driven reflection mechanism enables the co-evolution of benchmark function code and their landscape characteristics with minimal human intervention, providing interpretable design insights.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.