Reproduce “HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ru Peng, Tianyu Zhao, Xijun Gu, Zhiting Fan, Haokai Xu, Jinyang Zhang, Yawen Zeng, Yihong Zhuang and 4 more
HSS-Synth builds training data for open-ended humanities and social science tasks from screened texts across 14 fields. It derives requests from each source, checks their fit, and anchors answers in the source. Qwen3-8B-Base trained on it scores 57.84 on an eight-capability average, above listed synthesis baselines but below instruction-tuned Qwen3-8B; the average is not a single benchmark or evidence of balanced field coverage.
Key insights
- A structured, subject-centric approach with a defined domain system is crucial for effective and comprehensive data synthesis in complex, open-ended domains like HSS.
- Incorporating mechanisms like Multi-Attribute Instruction Backtranslation and Q&A Alignment Checks significantly enhances the diversity and fidelity of generated instructions.
- Teacher-Forced Answering (TeachForceA), by using seed documents as semantic anchors, effectively overcomes inherent LLM generation limitations, leading to more factual, complete, and stylistically consistent synthetic answers.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.