Reproduce “Scalable and Domain-General Abstractive Proposition Segmentation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mohammad Javad Hosseini, Yang Gao, Tim Baumgärtner, Alex Fabrikant, Reinald Kim Amplayo

A method for scalable and domain-general Abstractive Proposition Segmentation uses supervised training and knowledge distillation to create efficient models. This approach, evaluated with NLI-based metrics, enables smaller Gemma models to achieve performance comparable to larger Gemini teacher models across diverse text domains, improving upon few-shot methods in recall and overall quality.

Key insights

  • Supervised training of LLMs for APS substantially outperforms few-shot prompting, effectively addressing the issue of under-extraction and improving overall quality.
  • Knowledge distillation, leveraging large amounts of synthetically generated multi-domain data, successfully transfers advanced APS capabilities from large teacher models (Gemini) to smaller, more efficient student models (Gemma).
  • The NLI-based evaluation metrics correlate well with human judgments, providing a robust and objective means to measure proposition quality, including atomicity and self-containedness.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.