Reproduce “Bringing Structure into Summaries: a Faceted Summarization Dataset for Long Scientific Documents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Rui Meng, Khushboo Thaker, Lei Zhang, Yue Dong, Xingdi Yuan, Tong Wang, Daqing He

This paper presents FacetSum, a new large-scale dataset for faceted summarization of scientific documents, comprising over 60,000 articles with author-provided structured abstracts. The dataset enables the training and evaluation of models designed to generate structured summaries, demonstrating that explicit facet-specific training improves performance while also highlighting current neural models' limitations in processing very long input documents.

Key insights

  • Different abstract facets (e.g., Purpose, Findings) consistently align with distinct logical sections of the full scientific paper, empirically validating the dataset's structured nature.
  • BART models trained to generate specific facets (BART-Facet) significantly outperform models attempting to generate a concatenated full abstract, indicating the importance of explicit structural guidance.
  • While full papers contain more information, current generation models perform better when summarizing from shorter inputs like 'Introduction + Conclusion' due to challenges in processing extremely long documents.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.