Reproduce “Authorization Drift in LLM Agents: Evaluating Boundary Compliance in Multi-Step Tool Use” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chuxue Cao, Jingru Zeng, Boqin Yin, Yujin Zhou, Yidan Huang, Sujin Chen, Jinluan Yang, Haoran Li and 2 more
The authors introduce ADB, a 648-sample benchmark across seven domains testing whether tool-using agents take tempting but unnecessary unauthorized actions when a compliant path exists. Across 13 models, higher task success accompanies higher detected drift. At least one clean successful trajectory was observed on 555 of 644 tasks, evidence of reachability on most tasks, not proof every model can succeed cleanly.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.