Reproduce “RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
ZhuoXin Liu, Zhiming Ma, Ying Zhang, Mengzheng Yang, Yifan Wang, Zhengqi Huang, Yanhan Zhou, Zekun Lin and 5 more
RiskChainBench evaluates platform-risk investigation as a linked pipeline: first restoring obfuscated messages and recovering the intended destination, then browsing a controlled website and producing an evidence-supported risk judgment. Across 10 models, the best systems reached 95.22% Entry Top-1 restoration accuracy and 62.8% web-only decision accuracy, while entry gating exposed large end-to-end losses for models that struggled to recover the correct destination.
Key insights
- Character-level restoration quality does not reliably predict actionable routing: models with low character error can still select the wrong destination.
- Web-only and end-to-end results can diverge sharply; Qwen3.6 Plus lost 32.3 percentage points and Kimi K2.5 lost 29.5 points after entry gating.
- Execution reliability was the largest pooled first-failure category at 31.9%, indicating that navigation, interaction control, and evidence collection remain major bottlenecks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.