Reproduce “From Latent Space to Jacobian Space: Measuring, Evading, and Training Against Safety-Content Accessibility” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mohammad Mosafer
The authors introduce pre-generation safety evaluation separating refusal behavior from danger content accessible in a model’s internal state. A Jacobian-space readout measures vocabulary content, while a transport-amplification profile relates it to a latent safety direction. In a single-seed Qwen2.5-0.5B DPO experiment, refusal rose from 0.05 to 0.96 as SafetyAUC fell from 0.728 to 0.541, a chance-level result; this does not establish a general tuning effect.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.