Reproduce “HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Peilin Wu, Mian Zhang, Kun Wan, Wentian Zhao, Kaiyu He, Xinya Du, Zhiyu Chen
HiPRAG presents a framework that uses hierarchical process rewards to train agentic RAG models, enabling fine-grained optimization of search decisions. This approach significantly reduces suboptimal search behaviors like over-search and under-search, leading to improved answer accuracy and efficiency.
Key insights
- Optimizing the intermediate reasoning and retrieval process at a fine-grained level, rather than solely on final outcomes, is effective for improving agent reliability and efficiency.
- A hierarchical reward design can effectively balance multiple objectives (format adherence, final correctness, process efficiency) by gating process-level bonuses.
- Direct, on-the-fly detection of suboptimal search actions using external LLM judges provides robust and practical feedback signals for RL training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.