Reproduce “Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Praneeth Narisetty, Shiva Nagendra Babu Kore, Uday Kumar Reddy Kattamanchi, Jayaram Kumarapu
LaunchSafe Research conducted an adaptive evaluation of 'out-of-band' defenses against prompt injection in LLM agents, finding that the Progent defense significantly reduced attack success rates from 25.8% to 4.2% even against standard attacks and maintained this robustness against a custom-crafted adaptive attack. This resilience contrasts with the historical failures of 'in-band' detection methods.
Key insights
- Prompt injection is fundamentally a 'control/data confusion' vulnerability, which has historically been best mitigated by structural separation and deterministic enforcement rather than detection.
- Out-of-band defenses, which employ deterministic gates at the action layer, represent a more robust paradigm for protecting agent action integrity compared to model-layer detection methods.
- Despite their strengths, out-of-band defenses still face open challenges, including under-specified provenance assignment, securing 'in-the-loop' tasks, mitigating text-to-text harms, and addressing implicit flows or side channels.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.