Reproduce “WildSci: Advancing Scientific Reasoning from In-the-Wild Literature” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tengxiao Liu, Deepak Nathani, Zekun Li, Kevin Yang, William Yang Wang
Researchers at the University of California, Santa Barbara developed WildSci, an automated pipeline and dataset comprising 56,000 multiple-choice questions derived from peer-reviewed scientific literature to improve large language model reasoning in scientific domains. Models trained on WildSci demonstrated an average increase of 7.26% on established science benchmarks for the Qwen2.5-1.5B-Instruct model, indicating enhanced generalization capabilities.
Key insights
- High-quality, research-level scientific reasoning data can be automatically and scalably synthesized directly from "in-the-wild" peer-reviewed literature.
- Reinforcement Learning with Verifiable Rewards (RLVR) can be effectively adapted for complex scientific reasoning tasks by framing them into multiple-choice questions with clear reward signals.
- The refinement stage, particularly expanding answer choices and removing superficial cues, is crucial for increasing question difficulty and enhancing model generalization to out-of-domain benchmarks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.