Reproduce “Zero-shot Clarifying Question Generation for Conversational Search” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhenduo Wang, Yuancheng Tu, Corby Rosset, Nick Craswell, Ming Wu, Qingyao Ai
A zero-shot system generates clarifying questions for conversational search, addressing the cold start problem and data bias by leveraging a constrained GPT-2 model. The approach, employing facet-constrained generation and multiform prompting with WSDM ranking, achieved 82.6% "Good" naturalness and 68.9% "Good" usefulness in human evaluations, outperforming finetuned baselines.
Key insights
- Treating clarifying question generation as a facet-constrained language generation problem, where facet words act as explicit decoding constraints, significantly improves the relevance and specificity of generated questions.
- Using clarifying question templates as prompts effectively guides pre-trained language models to produce output in the desired question format, overcoming their tendency to generate narrative text.
- The 'question-body evaluation' strategy provides a more accurate assessment of a model's true generative capacity by mitigating bias from template overlaps in automatic metrics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.