Reproduce “Telling FIBS: Probes Effectively Detect Sabotage and Catch Unverbalized Deception” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Oskar J. Hollinsworth, Alex F. Spies, Tigist Diriba, Adam Gleave, Chris Cundy
The authors train probes on internal activations to monitor hidden side-task actions by tool-using agents using contrast scores from prefilled “Yes” and “No” answers. On matched BashArena steps for Qwen3.5-397B-A17B, EFC reaches 0.980 AUROC and 0.73 recall, versus Haiku’s 0.976 and 0.35. This is in-sample evidence at a 1% false-positive rate, not a held-out guarantee.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.