Reproduce “A$^2$Search: Ambiguity-Aware Question Answering with Reinforcement Learning” on OpenResearch

About this paper

Fengji Zhang, Xinyao Niu, Chengyang Ying, Guancheng Lin, Zhongkai Hao, Zhou Fan, Chengen Huang, Jacky Keung and 2 more

A2SEARCH introduces an annotation-free, end-to-end reinforcement learning framework that enables large language models to identify ambiguous questions and generate multiple valid answers. The approach achieves new state-of-the-art performance on multi-hop QA benchmarks, with A2SEARCH-7B yielding an average AnsF1@1 score of 48.4% and demonstrating superior inference efficiency with a single greedy rollout compared to larger baselines.

Key insights

• A substantial portion of questions in existing QA benchmarks are inherently ambiguous (e.g., 27.6% in MuSiQue), indicating a fundamental mismatch between current QA paradigms and real-world information complexity.
• Automated data generation for alternative answers is a scalable and effective alternative to manual annotation, overcoming previous bottlenecks in creating ambiguity-aware training datasets.
• Training LLMs with explicit ambiguity awareness and a tailored multi-answer reward function leads to more robust, efficient, and human-aligned QA systems capable of navigating nuanced language.

Summary by alphaXiv.