Reproduce “Aria: An Agent For Retrieval and Iterative Auto-Formalization via Dependency Graph” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hanyu Wang, Ruohan Xie, Yutong Wang, Guoxiong Gao, Xintao Yu, Bin Dong

ARIA is an auto-formalization agent designed to translate informal mathematical statements into formal Lean code by using a Graph-of-Thought planning process, retrieval-augmented generation, and compiler-guided reflection. The system demonstrated a 42.9% final accuracy on homological conjectures, a dataset where other models scored 0%, and improved semantic checking with its term-level grounding module.

Key insights

  • Structured planning via a Graph-of-Thought (GoT) enables LLMs to recursively decompose complex statements, synthesize novel definitions, and manage conceptual dependencies more effectively than one-pass generation.
  • Integrating a compiler-in-the-loop reflection mechanism allows for iterative refinement of generated formal code, ensuring syntactic correctness and reducing propagation of errors.
  • Augmenting semantic checking with term-level grounding, where authoritative formal definitions are used to contextualize LLM reasoning, is essential for identifying subtle semantic mismatches that textual comparisons miss.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.