Reproduce “Getting pwn'd by AI: Penetration Testing with Large Language Models” on OpenResearch

About this paper

Andreas Happe, Jürgen Cito

Researchers from TU Wien demonstrated that large language models can act as "AI sparring partners" for penetration testers, generating realistic attack plans and successfully exploiting vulnerabilities to gain root privileges in a controlled environment. The study found LLMs could identify common privilege escalation vectors and suggest exploitation examples, highlighting their potential to augment human cybersecurity capabilities.

Key insights

• Large language models can formulate realistic, high-level penetration testing plans and identify specific attack vectors, demonstrating strategic planning capabilities.
• LLMs can actively engage in low-level vulnerability hunting and exploitation within a live system, routinely achieving root privileges through common attack paths.
• While LLMs exhibit indications of reasoning, their vulnerability identification and exploitation suggestions are largely based on pattern-matching from training data rather than a deep understanding of system mechanics.
• Built-in ethical moderation in LLMs can be circumvented through prompt engineering, raising concerns about the dual-use potential and security implications of such tools.

Summary by alphaXiv.