Reproduce “Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zimo Ji, Congying Xu, Zongjie Li, Yudong Gao, Xin Wei, Shuai Wang, Shing-Chi Cheung

This research demonstrated that current static scanners for LLM agent skills are highly susceptible to semantics-preserving evasions, with developed techniques achieving bypass rates up to 99.8%. A behavior-centric runtime auditor, SKILLDETONATE, was then proposed, effectively detecting over 94% of original and evasive malicious skills at a 2% false-positive rate.

Key insights

  • Static, appearance-based auditing mechanisms for LLM agent skills are fundamentally inadequate against adaptive attackers who can alter malware's visible form without changing its malicious behavior.
  • A behavior-centric runtime auditing paradigm is crucial for robustly detecting evasive and multi-stage LLM agent skill malware by focusing on observable effects rather than static signatures.
  • Effective dynamic analysis for LLM agents requires specialized techniques such as tracking dynamically generated natural language instructions and tracing sensitive data flow across diverse information mediums.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.