Reproduce “Games for AI Control: Models of Safety Evaluations of AI Deployment Protocols” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Charlie Griffin, Louis Thomson, Buck Shlegeris, Alessandro Abate
AI-Control Games formalize the evaluation of deployment protocols for potentially deceptive AI systems as partially observable, two-objective stochastic games. Applied to trusted monitoring of an untrusted coding assistant, the framework computes worst-case safety–usefulness trade-offs and identifies protocol improvements, including audit rates above the available average budget and randomized strategies when monitoring quality is weak.
Key insights
- The formalism makes explicit that AI-Control evaluation is a leader–follower problem: Blue chooses a protocol, while Red selects behavior that minimizes safety, with usefulness measured in the honest mode.
- Using the full nominal audit budget at every point is not necessarily optimal; exhausting audits earlier can improve worst-case protection over a finite sequence of tasks.
- When the trusted monitor is less discriminative, randomized protocols can become important, whereas deterministic strategies are more common near the frontier under a stronger monitor.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.