Reproduce “GigaEvo: An Open Source Optimization Framework Powered By LLMs And Evolution Algorithms” on OpenResearch

About this paper

Valentin Khrulkov, Andrey Galichin, Denis Bashkirov, Dmitry Vinichenko, Oleg Travkin, Roman Alferov, Andrey Kuznetsov, Ivan Oseledets

GigaEvo is an open-source framework that merges Large Language Models (LLMs) with evolutionary algorithms to tackle complex optimization challenges, providing a transparent and extensible platform for research. The framework reproduces and in several cases improves upon state-of-the-art results across geometric optimization, algorithmic design, and prompt/agent evolution benchmarks.

Key insights

• An open-source, modular, and transparent framework is crucial for fostering reproducibility and accelerating research in LLM-evolution hybrid systems.
• Enriching LLM mutation prompts with contextual information, including generated insights from program analysis and bidirectional lineage tracking (ancestor innovations and descendant improvements), enhances the evolutionary search process.
• The heterogeneous use of different LLM models for specific sub-tasks can be beneficial, leveraging their unique strengths for various problem types (e.g., geometric vs. discrete optimization).
• Rewrite-based mutation strategies proved more robust for open-source LLMs compared to diff-based methods in code generation for evolutionary search.

Summary by alphaXiv.