Reproduce “HPFA: Hypergraph-Based Paired Failure Attribution for LLM Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Runchuan Zhu, Hongbin Lai, Bowen Jiang, Junrui Zhang, Zhangheng LI, Ostap Kilbasovych, Junyuan Hong
HPFA (Hypergraph-based Paired Failure Attribution) is a framework that efficiently localizes root causes of Large Language Model (LLM) reasoning failures by leveraging paired successful and failed trajectories alongside hypergraph modeling of reasoning dependencies. This method generates data to train a lightweight attributor model, which improves LLM reflective reasoning and task accuracy during test-time inference across diverse complex tasks.
Key insights
- Utilizing a successful reference trajectory alongside a failed one dramatically improves the efficiency of root cause localization by focusing the search on divergent reasoning steps.
- Representing reasoning traces as hypergraphs effectively captures non-linear logical dependencies, enabling more accurate identification of true upstream root causes, especially in longer and more complex tasks.
- High-quality, verified failure attribution data generated by HPFA can train a compact attributor model that significantly enhances LLMs' ability to self-correct and improve performance on new, unseen reasoning failures at inference time.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.