Reproduce “DeepRepoQA: Code Repository Question Answering with Deep Agent Exploration” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Weihan Peng, Yuling Shi, Yingwei Ma, Longfei Yun, Beijun Shen, Xiaodong Gu
DEEPREPOQA introduces a framework for code repository question answering that uses an LLM agent guided by Monte-Carlo Tree Search (MCTS) to perform deep, verifiable, multi-hop reasoning. The system achieved an overall score of 70.06 on the SWE-QA benchmark with strong LLM backbones, outperforming existing open-source methods and demonstrating performance comparable to commercial tools in correctness, completeness, and reasoning.
Key insights
- The MCTS framework, empowered by LLM feedback and memory, enables principled multi-hop inference and systematic exploration across code repositories, outperforming greedy single-trajectory search designs.
- The Evaluation Agent's learned value estimation for action-observation pairs is crucial for guiding path selection within MCTS, significantly contributing to performance by efficiently replacing expensive traditional multi-step rollouts.
- A comprehensive state summarization by the Perception Agent and a semantically meaningful action space enhance the efficiency and effectiveness of deep repository analysis.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.