Reproduce “Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Anikait Singh, Teresa Zhang, Yoonho Lee, Roshen Sanjay Nair, Aviral Kumar, Chelsea Finn
Librarian Agents prepares a reusable hierarchical workspace of synthetic files linked to source documents, then uses a separate agent to answer later questions. The builder is trained with reinforcement learning using downstream answer quality. On BrowseComp-Plus, it improves accuracy over baselines by an average of 33 percentage points across two fixed answering models; this result measures answer accuracy, not cost.
Key insights
- The workspace is judged by whether it helps a separate model answer questions, rather than by summary quality alone.
- Learned organization can form richer cross-document hierarchies than an untuned builder, and raw-source links help recover details omitted from summaries.
- Accuracy gains do not automatically mean lower inference cost: the trained workspace can require more reads and tokens per query, so reuse frequency matters.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.