Reproduce “Generating Query-Relevant Document Summaries via Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Nitin Yadav, Changsung Kang, Hongwei Shang, Ming Sun

The ReLSum framework introduces a reinforcement learning approach to generate concise, query-relevant summaries of product descriptions for e-commerce search. This method explicitly optimizes summary generation to improve relevance for a downstream ranking model, resulting in statistically significant lifts in online add-to-cart actions, units, and orders.

Key insights

  • Reinforcement learning can effectively align LLM-based summarization with the specific objective of improving downstream search relevance by treating a pre-trained ranker as a reward signal.
  • Targeting training on examples where product descriptions meaningfully influence relevance enhances the efficiency and effectiveness of the LLM in extracting crucial attributes for query matching.
  • Generating query-relevant summaries via an RL-trained LLM provides a practical solution to incorporate rich product context into real-time ranking systems, circumventing the latency constraints of full descriptions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.