Reproduce “RAS: Measuring LLM Safety Through Refusal Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chang-Chieh Huang, Yan-Lun Chen, Chia-Mu Yu, Wei-Bin Lee
Researchers from National Yang Ming Chiao Tung University and Hon Hai Research Institute developed SafeVec, a white-box methodology, to measure large language model safety by quantifying internal "refusal alignment" using a Refusal Alignment Score (RAS). This representation-level approach is 210 times faster than traditional output-based evaluations and correlates with output-level safety metrics across Llama, Gemma, and Qwen model families.
Key insights
- Safety-aligned LLMs exhibit distinct internal shifts along refusal-related directions within their middle-to-late decoder layers when processing harmful requests.
- Internal refusal alignment signals are generalizable across diverse LLM architectures (e.g., Llama, Gemma, Qwen), though family-specific calibration is necessary for score comparability.
- Representation-level safety signals can reliably indicate a model's propensity for harmful output, offering an early and efficient diagnostic alternative to behavioral evaluation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.