Reproduce “Quantifying stability of non-power-seeking in artificial agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Evan Ryan Gunter, Yevgeny Liokumovich, Victoria Krakovna
The paper analyzes whether shutdown guarantees persist when an agent’s Markov decision process (MDP) environment changes. It proves that sufficiently small Hausdorff bisimulation perturbations preserve a shutdown-time bound for near-optimal policies, provided safe states are isolated, and bounds local decreases in eventual-shutdown probability for fixed smooth policies. These conditional mathematical results are not deployment evidence, and eventual shutdown permits temporary resistance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.