Reproduce “Multi-hop Reading Comprehension through Question Decomposition and Rescoring” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sewon Min, Victor Zhong, Luke Zettlemoyer, Hannaneh Hajishirzi

DECOMPRC, a system from the University of Washington and Allen Institute for AI, addresses multi-hop reading comprehension by decomposing complex questions into simpler sub-questions and employing an answer-aware rescoring mechanism. This approach achieved state-of-the-art performance on the HotpotQA dataset, scoring 70.57 F1 in the distractor setting and 43.26 F1 in the full wiki setting, while demonstrating enhanced robustness against adversarial inputs.

Key insights

  • Decomposing complex multi-hop questions into simpler, single-hop sub-questions simplifies the overall reasoning task, making it more manageable for current RC models.
  • Rethinking sub-question generation as a span prediction problem significantly reduces the necessary supervised data, enhancing data efficiency for complex linguistic operations.
  • An answer-aware rescoring mechanism, which considers both sub-questions and their derived answers and evidence, is crucial for effectively selecting the optimal reasoning path and mitigating cascading errors.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.