Reproduce “Don't Guess, Ask Wisely: Benchmarking and Training Proactive Elicitation in Open-Domain LLM Conversations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rongyu Zhang, Mutian Bao, Jinjian Zhang, Yuetian Du, He Xu, Wei Zhou, Xiaojun Zhu, Ming Kong and 3 more
The authors introduce proactive elicitation, training models to ask targeted questions that recover missing user intent and details before answering. They build VagueBench with annotated hidden information and train DialogGRPO using immediate information gain and lookahead value. On intent-vague queries, Qwen3.5-27B reaches 88.4% Ask%, 0.64 coverage F1, and 0.76 IAR, exceeding trajectory-level SFT; evaluation uses cooperative simulated users, not live conversations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.