Reproduce “A Deep Reinforcement Learning Approach for Interactive Search with Sentence-level Feedback” on OpenResearch

About this paper

Jianghong Zhou, Joyce C. Ho, Chen Lin, Eugene Agichtein

This paper introduces DQrank, a Deep Reinforcement Learning model, to enhance interactive search by effectively incorporating fine-grained sentence-level user feedback. The system demonstrates at least 12% nDCG@10 and MRR improvement over previous state-of-the-art RL approaches on multiple datasets by addressing challenges of action space and data scarcity.

Key insights

• Leveraging sentence-level feedback through a dynamic Reinforcement Learning framework (DQrank) significantly outperforms static BERT-based ranking, demonstrating the value of sequential decision-making.
• Components like Data Augmentation and State Retrieval synergistically improve performance, boosting initial scores and accelerating convergence in online sessions.
• Rearrangement Learning is crucial for aligning the user simulation function's relevance scores with the Q-function's optimal actions, preventing sub-optimal results.

Summary by alphaXiv.