Reproduce “PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Features” on OpenResearch

About this paper

Wei Zou, Yupei Liu, Yanting Wang, Ying Chen, Neil Gong, Jinyuan Jia

Researchers from Pennsylvania State University and Duke University introduce PIShield, a prompt injection detection method that exploits the intrinsic ability of instruction-tuned Large Language Models to encode distinct internal signals for instructional content. PIShield demonstrates high detection efficacy against various attacks and maintains low false positives across diverse benchmarks, all while being computationally efficient by avoiding full LLM inference at test time.

Key insights

• Instruction-tuned LLMs inherently produce linearly separable internal representations (specifically, last-token residual-stream vectors) that can distinguish between purely data-driven content and content containing executable instructions.
• The instruction-tuning process is crucial for an LLM to develop these discriminative internal representations, as non-instruction-tuned base models do not exhibit such clear signals for prompt injection detection.
• Optimal detection performance often comes from intermediate or later layers of the LLM, rather than exclusively the final layer, suggesting these layers effectively capture general 'instructional intent'.

Summary by alphaXiv.