Reproduce “Plans They Abandon, Reports They Author: The Narrative Layer of Autonomous Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Obada Kraishan, Kulsawasd Jitkajornwanich
The authors compare coding-agent plans, tool logs, and self-reports by mapping them to action types. Across 5,115 sessions, reports referred to 9.4% of logged actions. A report-only model reconstructed action types with mean F1 of 0.202, the harmonic mean of precision and recall, not a reader study. Omission was conservative; human correction changed it by 1.6 percentage points. The study uses one corpus.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.