Reproduce “AFCS: Atomic-Fact-Conditioned Synthesis for Factual Knowledge Pretraining” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yunhao Wang, Feng Zhang, Tinghao Yu, Xing Wu, Jiang Zhou
AFCS makes source-supported facts explicit targets for synthetic-text generation, extracting and verifying facts before conditioning generation on each fact with local context. On English Wikipedia-derived FineWiki, it achieves 85.1% source-fact coverage. After 20B tokens of continued pretraining, a 7B model reaches 29.80% PopQA accuracy versus 26.42% for Nemotron-CC; coverage does not establish model acquisition.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.