Reproduce “Combining Textual and Structural Information for Premise Selection in Lean” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Job Petrovčič, David Eliecer Narvaez Denis, Ljupčo Todorovski

Researchers developed a graph-augmented approach for premise selection in Lean 4 by integrating a heterogeneous dependency graph with text embeddings using a Relational Graph Convolutional Network. This method achieved a 36.44% relative improvement in Recall@1, reaching 18.31%, over the ByT5-small ReProver baseline on the LeanDojo Mathlib benchmark.

Key insights

  • Integrating a fine-grained heterogeneous dependency graph with language models can significantly enhance premise selection performance by leveraging structural relationships that complement textual information.
  • Relational Graph Convolutional Networks are effective in propagating structural context within formal mathematics libraries, leading to more informed embeddings for both proof states and premises.
  • While the graph structure improves performance, an ablation study suggests that aspects of the training paradigm, such as comprehensive negative sampling and InfoNCE loss, may also contribute substantially to the observed gains, indicating areas for further investigation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.