Reproduce “guardscope: Measuring Which Actions Runtime Controls Check in LLM Agent Systems” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kentaroh Toyoda, Alex Leung, Bhuvaneswari U
The authors introduce guardscope, a framework that predicts paths runtime controls can check from source code and compares control records with kernel observations. In a tested T5 deployment, harness permission checks plus a gateway mediated both tool-call classes, while eight other actions took effect without policy evaluation. The workload exercised four of 15 defined paths; checking does not establish policy correctness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.