Reproduce “Sci2Pol: Evaluating and Fine-tuning LLMs on Scientific-to-Policy Brief Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Weimin Wu, Alexander C. Furnas, Eddie Yang, Gefei Liu, Akhil Pandey Akella, Xuefeng Song, Dashun Wang, Han Liu

The authors introduce Sci2Pol-Bench and Sci2Pol-Corpus to evaluate and train models to turn scientific papers into policy briefs. They assemble citation-linked policy documents, screen them for relevance, polish selected examples into brief style, and fine-tune models. Gemma-3-27B-SFT reports a 73.43 average versus DeepSeek-V3’s 73.35, a narrow, untested lead on this benchmark, not evidence of general superiority.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.