Reproduce “AgentTrails: Towards Trust and Reuse for Agentic Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Eden Wu, Sonia Castelo, Yurong Liu, Cláudio T. Silva, Juliana Freire
New York University researchers developed AgentTrails, a visual analytics system that transforms raw, chronological large language model agent execution logs into explicit, auditable provenance graphs. This system effectively reveals hidden data dependencies, aligns varied execution paths, and identifies recurring computational motifs in real-world scientific tasks.
Key insights
- Understanding agent behavior is enhanced by framing it around auditable provenance graphs rather than simple chronological logs, exposing hidden data dependencies and workflow structures.
- Implicit dataflow dependencies within agent traces can be effectively reconstructed post-hoc using a combination of exact, semantic, and LLM-refined evidence, without requiring runtime agent instrumentation.
- A 'joined quotient graph' approach, derived from clustering abstracted activity capsules, provides a robust method to align and compare multiple stochastic agent executions, identifying shared workflows and divergent paths.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.