Reproduce “How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring” on OpenResearch

About this paper

Yang Gao

This research evaluates the reliability of automated judges, including dedicated classifiers and LLM-as-judges, used for scoring Attack Success Rates (ASRs) in large language model jailbreak evaluations. It reveals that these judges exhibit systematic biases, such as over-flagging or under-flagging, and are vulnerable to content-preserving adversarial manipulations, which can lead to misleading ASR measurements.

Key insights

• Automated judge families exhibit systematic and contrasting biases: dedicated classifiers show high recall but lower precision (over-flagging), while LLM-as-judges have high precision but erratic, low recall (under-flagging).
• LLM-as-judges are highly fragile to simple content-preserving wrappers, with flip rates up to 100%, suggesting reliance on superficial cues rather than semantic harm.
• Dedicated classifiers, despite robustness to simple wrappers, are vulnerable to white-box gradient-based adversarial attacks, showing a 70% flip rate with a small optimization budget.

Summary by alphaXiv.