Reproduce “Boosting legal case retrieval by query content selection with large language models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Youchao Zhou, Heyan Huang, Zhijing Wu

Researchers at Beijing Institute of Technology developed a method using large language models to reformulate legal case queries by selecting salient content, which consistently improved the retrieval performance of both sparse and dense legal case retrieval models. LLM-generated summaries proved most effective, boosting BERT-PLI's P@5 by +4.67% and MAP by +5.69%, while also providing insights into how current retrieval models perceive legal query relevance compared to human experts.

Key insights

  • Retrieval models, both sparse and dense, perceive only a portion of salient content in legal queries and show a disconnect from human legal judgment, each prioritizing distinct textual features.
  • Large Language Models can effectively condense complex legal queries into their most salient components, improving query quality for retrieval without domain-specific fine-tuning.
  • LLM-generated query summaries, which maintain fluent natural language, generally lead to better retrieval performance compared to keyword or key sentence extractions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.