Reproduce “When Bots Take the Bait: Exposing and Mitigating the Emerging Social Engineering Attack in Web Automation Agent” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xinyi Wu, Geng Hong, Yueyue Chen, MingXuan Liu, Feier Jin, Xudong Pan, Jiarun Dai, Baojun Liu

Researchers from Fudan University, Zhongguancun Laboratory, Shanghai Innovation Institute, and Tsinghua University systematically exposed social engineering attacks (AGENTBAIT) against web automation agents, finding an average 67.5% attack success rate across mainstream frameworks. They also developed SUPERVISOR, a runtime defense that reduced attack success by 78.1% with minimal overhead.

Key insights

  • Web agents exhibit "cognitive blind spots" analogous to human psychological vulnerabilities, making them susceptible to social engineering cues like authority trust bias and context consistency illusion.
  • Existing intrinsic safety prompts in agents are insufficient to reliably judge risks from dynamic webpage contexts, especially against implicit malicious actions like permission abuse.
  • A defense mechanism needs to perform joint consistency verification at runtime, checking both perceived environmental factors and intended actions against agent background knowledge and user goals.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.