Reproduce “Encrypted Skills Can Become New Security Exploits If AI Agents Know How to Decipher” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yan-Lun Chen, Pin-Yu Chen, Chia-Mu Yu

Ciphered Skill Injection (CSI) tests whether agents decode and follow instructions hidden in third-party skills. It evaluates Caesar and layered attacks across eight models, measuring instruction recovery separately from attack attempts and verified goal achievement. In the main setting, peak attack-success rates were 47.5% for Caesar and 40.3% for layered CSI; results used shortened, rewritten skills and varied across models.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.