Reproduce “Self-Adapting Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Adam Zweiger, Jyothish Pari, Han Guo, Ekin Akyürek, Yoon Kim, Pulkit Agrawal

SEAL trains language models to generate self-edits that improve adaptation to new information or tasks. It applies generated training examples or update settings through finetuning, then uses downstream performance to train the edit-generation policy. In a single-passage knowledge test, SEAL reaches 47.0% accuracy, above passage-only finetuning and base-model synthetic data, but it does not lead in every training regime.

Key insights

  • LLMs can be trained to autonomously generate effective finetuning data and adaptation strategies, transforming them from passive data consumers into active, self-improving learners.
  • Optimizing the *generation* of synthetic training data through RL can lead to more effective model updates than using fixed prompting strategies or even data from larger, off-the-shelf proprietary models like GPT-4.1.
  • The framework provides a foundational step towards continuous learning and the development of agentic systems capable of structured self-modification and incremental knowledge acquisition.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.