Reproduce “Large Language Model-Based Automatic Formulation for Stochastic Optimization Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Amirreza Talebi
This research introduces a systematic approach for Large Language Models to automatically formulate stochastic optimization problems from natural language descriptions. The study demonstrates that while current LLMs do not achieve perfect solutions, structured and multi-agent prompting significantly improves their ability to generate partially correct mathematical models, evaluated using a novel soft scoring metric.
Key insights
- Despite no LLM achieving full accuracy (producing perfectly executable code with correct optimal objective values), significant levels of partial correctness were observed across generated SO models.
- Structured prompting strategies, such as `cot_instructions` and the `agentic` multi-agent framework, consistently and significantly improved LLM performance in terms of partial scores and matching components.
- The introduced soft scoring metric is crucial for dissecting LLM performance in complex formulation tasks, offering granular feedback on specific strengths and weaknesses in generating different model components.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.