Reproduce “Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answers” on OpenResearch

About this paper

Tianhua Zhang, Kun Li, Hongyin Luo, Xixin Wu, James Glass, Helen Meng

The Adaptive Query Rewriting (AdaQR) framework enables training effective conversational query rewriters with limited rewrite annotations and no passage labels. It leverages the marginal probability of conversational answers as a weak supervision signal, yielding robust in-domain and out-of-domain retrieval performance across various datasets and retriever types.

Key insights

• The marginal probability of conversational answers effectively serves as an accessible and cost-efficient weak supervision signal to model retriever preferences, obviating the need for expensive passage relevance labels.
• Preference alignment via DPO, even under weak supervision, significantly enhances the rewriter's ability to generate more useful queries for downstream retrieval, improving both in-domain and crucial out-of-domain performance.
• Compact LLMs, when combined with limited supervised fine-tuning and subsequent weak preference optimization, can achieve state-of-the-art conversational retrieval performance with substantially reduced annotation overhead.

Summary by alphaXiv.