Reproduce “Scalable Influence and Fact Tracing for Large Language Model Pretraining” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tyler A. Chang, Dheeraj Rajagopal, Tolga Bolukbasi, Lucas Dixon, Ian Tenney
Researchers at Google DeepMind developed TrackStar, a scalable gradient-based influence method, to trace the impact of pretraining data on large language models. The method successfully attributed the outputs of an 8B-parameter LLM to a corpus of over 160 billion tokens without subsampling, revealing that examples directly entailing a fact are not always the most causally influential on a model's prediction.
Key insights
- A significant 'misalignment' was identified between factual attribution (finding examples that entail a fact) and causal influence (finding examples that primarily shape a prediction), where model-agnostic methods excel at the former but TrackStar at the latter.
- As LLMs increase in scale and training data, their influential examples align more closely with factually entailing data, suggesting a more grounded learning process for larger models.
- Many influential examples do not directly state a target fact but rather reinforce priors, provide partial matches, or offer information supporting multi-hop reasoning, indicating a complex knowledge acquisition mechanism.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.