Reproduce “ProxyPrompt: Securing System Prompts against Prompt Extraction Attacks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhixiong Zhuang, Maria-Irina Nicolae, Hui-Po Wang, Mario Fritz
ProxyPrompt introduces a defense mechanism for Large Language Model (LLM) system prompts, optimizing a 'proxy' prompt in the LLM's embedding space to prevent extraction attacks while preserving the model's original task performance. The method achieved a 0 Approx-Match score and 94.70% semantic protection against prompt leakage, consistently maintaining a Utility-Ratio near 1.00 across diverse tasks and LLMs.
Key insights
- Optimizing prompt representations in the continuous embedding space offers a robust way to obfuscate sensitive instructions, making them hard to extract and semantically reconstruct.
- A dual-objective optimization, balancing utility preservation for legitimate use with disutility for extracted content, is crucial for effective and practical prompt defense.
- Evaluating prompt security requires semantic-level metrics (e.g., Semantic-Match) beyond traditional word-level overlaps, as attackers can use rephrasing to bypass simpler detection.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.