Reproduce “Prompt Engineering Through the Lens of Optimal Control” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yifan Luo, Yiming Tang, Chengfeng Shen, Zhennan Zhou, Bin Dong

This paper introduces an optimal control framework for multi-round interactions with Large Language Models, aiming to systematize Prompt Engineering from an empirical art into a mathematically grounded science. The framework unifies various existing prompting techniques and highlights the challenge of dynamically evolving action spaces in language models.

Key insights

  • A wide array of existing Prompt Engineering (PE) methods can be consistently reinterpreted and unified under an optimal control framework, offering a common language for analysis.
  • Multi-round PE involves a non-stationary 'action space' (prompt candidate set) that dynamically expands based on previous Large Language Model (LLM) responses, posing a unique challenge for traditional control theory.
  • The framework reveals how LLM stochasticity can be leveraged through ensemble methods and provides a structured view for multi-agent LLM collaborations as interacting control units.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.