Reproduce “PromptShield Home: Ambient Multimodal Prompt Injection Defense for Smart-Home Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
He Zhang, Feilong Li, Dingning Long, Yilin Cui, Peijun Zhang, Yuewen Zhang, Qianyao Xu, Xinyi Fu
A new safety concern, "Ambient Multimodal Prompt Injection," is defined for smart-home agents powered by Multimodal Large Language Models, where ambient signals are mistaken for user commands. The work presents the PromptShield-Home benchmark and a risk-aware decision framework, revealing that MLLMs tend to over-refuse valid commands, whereas traditional detectors over-execute on ambient signals, advocating for a hybrid system combining both for better safety and utility.
Key insights
- Traditional detectors and MLLMs exhibit opposite failure modes: detectors consistently over-execute on ambient signals (100% UER), while MLLMs systematically over-refuse legitimate commands (0% SCR), missing critical actions like falls.
- Aggregate accuracy is a misleading metric for safety-critical AI with skewed data distributions; dedicated metrics like Unsafe Execution Rate (UER) and Safe Completion Rate (SCR) are crucial for assessing safety and utility.
- A hybrid "route rather than replace" architectural approach, combining the complementary strengths of traditional detectors and MLLMs, is necessary for improved overall safety and utility in smart-home agents.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.