Reproduce “RLIRank: Learning to Rank with Reinforcement Learning for Dynamic Search” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jianghong Zhou, Eugene Agichtein
RLIRank proposes a deep reinforcement learning framework for dynamic search, incorporating a stacked LSTM for document ranking and an embedding-oriented Rocchio algorithm for query reformulation. The system demonstrated up to a 6.1% improvement in "alpha"-NDCG on TREC 2016 Dynamic Domain and over 20% improvement in nSDCG on TREC 2017 Dynamic Domain compared to existing baselines.
Key insights
- Reinforcement learning naturally supports sequential decision-making and integration of user feedback, which is crucial for dynamic search tasks with evolving information needs.
- A stacked LSTM trained in a stepwise manner effectively captures the evolving context of a search session by considering previously ranked documents and their features, leading to improved subsequent ranking decisions.
- Using dense word embeddings for query reformulation in a modified Rocchio algorithm allows for semantically rich and continuous adaptation of the query, enhancing the robustness and effectiveness of feedback integration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.