Reproduce “JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jianyi Hu, Hangtao Zhang, Yi Liu, Yeqi Zeng, Li Zeng, Xianlong Wang, Rui Wang, Leo Yu Zhang
The paper introduces JevAdvBench to test whether attacker-writable content can make typed decision models turn correct decisions into wrong ones. It evaluates black-box edits that add arguments, context, commands, or structural changes, while reviewing whether edits preserve the answer and controlling for repeated-call noise. On Jev, pooled attacks flip 46.3% of eligible decisions; this is benchmark-specific, not typical performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.