Reproduce “Auto-Configuring Scientific Simulators with Lightweight Coding-Agent Adapters” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Matthew Ho, Brian Liu, Jixuan Chen, Audrey Wang, Lianhui Qin
SIGA adapts an existing coding agent for scientific simulator setup by adding simulator-specific documentation retrieval, procedural memory, XML validation, and completion checks while leaving the model and general workflow unchanged. On ten held-out, harder GEOS tasks, its self-evolved adapter raises mean TreeSim from 0.720 to 0.789 across three runs per configuration. TreeSim measures structural agreement, not execution or physical correctness.
Key insights
- Off-the-shelf coding agents already possess many generic software manipulation capabilities, primarily lacking only the specific "simulator executable contract" to perform configuration tasks.
- A lightweight, external adapter design allows for leveraging existing robust coding agent behaviors while offering a more manageable and portable target for self-evolution.
- The dominant components of the adapter (e.g., validation, procedural memory, retrieval) can shift based on the specific interface challenges of different scientific simulators, demonstrating adaptability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.