Reproduce “PRACT: Optimizing Principled Reasoning and Acting of LLM Agent” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhiwei Liu, Weiran Yao, Jianguo Zhang, Rithesh Murthy, Liangwei Yang, Zuxin Liu, Tian Lan, Ming Zhu and 8 more
Salesforce AI Research developed PRAct (Principled Reasoning and Acting), a framework that guides LLM agents with explicit principles and automates principle refinement via Reflective Principle Optimization (RPO). This approach consistently improved performance over baselines across multiple environments and enabled smaller models like GPT-3.5-turbo to achieve performance comparable to GPT-4-turbo.
Key insights
- Explicit principles provide structured guidance for agents, improving decision-making, especially when encountering contradictory observations.
- Automated optimization of principles through RPO reduces manual prompt engineering and allows principles to evolve based on performance.
- Batching reflections (RPO-Batch) generally leads to more effective cross-trajectory reasoning and better performance than individual trajectory optimization.
- Action principles can help bridge capability gaps between different LLM sizes, enabling smaller models to achieve performance comparable to larger ones in some cases.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.