Reproduce “Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancement” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dingwei Chen, Zefang Zong, Zhipeng Ma, Leo Luo, Yang Li, Chengming Li, Peng Chen, Jie Jiang
AKBE trains question-answering agents to choose when to search by comparing current-policy answers with and without tools for each question, then reinforcing the correct, economical path. Across seven benchmarks, Qwen3-4B with AKBE improves average exact-match accuracy by 1.85 percentage points over Search-R1, reduces tool calls by 18%, and raises tool productivity by about 25% in Wikipedia-search question answering.
Key insights
- An agent's 'knowledge boundary' regarding tool necessity is dynamic and instance-specific, requiring on-policy probing rather than static assumptions for effective optimization.
- Providing fine-grained, category-specific supervision (e.g., for necessary tool use, redundant tool use, or hallucination-inducing tool use) is more effective than generic tool-call suppression.
- The method actively promotes knowledge internalization, where agents learn to rely on parametric knowledge when sufficient, reducing 'cognitive offloading' and fostering balanced tool reliance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.