Reproduce “Towards Rationality in Language and Multimodal Agents: A Survey” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Bowen Jiang, Yangxinyu Xie, Xiaomeng Wang, Yuan Yuan, Zhuoqun Hao, Xinyi Bai, Weijie J. Su, Camillo J. Taylor and 1 more

A survey from researchers at the University of Pennsylvania, Cornell, and Argonne National Laboratory proposes a four-axiom framework to define rationality in language and multimodal AI agents. It reinterprets existing advancements, demonstrating how various methods contribute to information grounding, logical consistency, invariance, and preference orderability, and identifies a critical lack of dedicated rationality evaluation metrics.

Key insights

  • The majority of current methods enhance agent rationality extrinsically, relying on external tools and systems rather than intrinsically altering the underlying model parameters.
  • Deliberative processes, including multi-round self-reflection for single agents and multi-agent collaboration, are effective strategies for improving logical consistency.
  • Multimodal foundation models, coupled with knowledge graphs and tool utilization, collectively strengthen information grounding and invariance to irrelevant context.
  • A critical lack of dedicated and comprehensive evaluation metrics specifically designed to measure agent rationality across the four proposed axioms persists in the field.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.