Reproduce “SkillJack: Persistent Skill Backdoors in Self-Evolving Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zonghao Ying, Xiangfan Wu, Huiyu Wu, Xing Zheng, Huangsheng Cheng, Xiaorong Shi, Jing Guo
Tencent Zhuque Lab's research unveils "SkillJack," a security vulnerability enabling self-evolving LLM agents to absorb and perpetually embed malicious behaviors into their reusable skill sets from poisoned experiences. This work shows that these stealthy, learned skills can bypass detection mechanisms, persist despite the deletion of original source data, and lead to unintended actions in subsequent tasks.
Key insights
- The process of skill compilation effectively "whitewashes" malicious content, significantly reducing its detectability by safety mechanisms compared to its raw, untransformed state.
- Malicious skills, once extracted, achieve "persistence isolation," meaning they remain active and able to trigger attacks even after their original source records have been removed from the agent's memory.
- These derived malicious skills gain "cross-layer promotion," allowing them to be retrieved and execute unintended actions on new tasks, including those not directly related to their original malicious intent.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.