Reproduce “Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available Evidence” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Owen Walters, Lihu Chen, Francesca Toni
The authors test whether search-augmented assistants use current evidence and cite it faithfully. In an example, an assistant searches for PayPal’s CEO, opens no page, and gives an outdated answer; Enrique Lores became CEO on 1 March 2026. In frozen-retrieval tests, stale answers and unfaithful citations were less frequent when pages were opened and cited, an association, not proof of causation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.