Reproduce “Steering Over-refusals Towards Safety in Retrieval Augmented Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Utsav Maskey, Mark Dras, Usman Naseem
Macquarie University researchers introduced RAGREFUSE, a benchmark to analyze LLM over-refusals in Retrieval-Augmented Generation (RAG) when faced with contaminated contexts. They also developed SAFERAG-STEERING, an embedding intervention that reduced over-refusal rates in benign queries from 53.4% to 4.3% for Llama-3.1-8B-Instruct while preserving legitimate refusals.
Key insights
- Over-refusal rates in RAG systems are highly sensitive to the presence and density of harmful content in retrieved contexts, even for benign user queries.
- LLMs exhibit non-uniform sensitivity to over-refusals across different domains, with Llama-3-8B-Instruct showing higher refusal rates in Chemical, Medical, and Legal contexts.
- Targeted, inference-time embedding interventions can effectively mitigate RAG-induced over-refusals on benign queries without compromising the model's ability to refuse genuinely harmful requests.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.