Reproduce “DynaAct: Large Language Model Reasoning with Dynamic Action Spaces” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xueliang Zhao, Wei Wu, Jian Guan, Qintong Li, Lingpeng Kong
DYNAACT introduces a three-stage computational framework for Large Language Models that dynamically constructs compact action spaces for sequential reasoning by leveraging submodular optimization. This method significantly enhances LLM performance across various reasoning benchmarks, achieving notable gains such as a 6.8% absolute improvement on the challenging MATH-500 dataset over prior strong baselines.
Key insights
- Framing action space construction as a submodular optimization problem effectively balances action utility and diversity, leading to information-dense and compact action sets that enhance reasoning.
- Learning effective action and state embeddings via a Q-learning objective is crucial for accurately assessing action relevance and guiding long-term sequential decision-making.
- A data-driven approach to constructing a comprehensive proxy action space allows for scalability and adaptability across diverse problem domains, overcoming limitations of manually defined or fixed action sets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.