Reproduce “Evaluation of Reinforcement Learning for Autonomous Penetration Testing using A3C, Q-learning and DQN” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Norman Becker, Daniel Reti, Evridiki V. Ntagiou, Marcus Wallum, Hans D. Schotten

Researchers from DFKI and ESA-ESOC evaluated Q-learning, Deep Q-Networks (DQN), and Asynchronous Advantage Actor-Critic (A3C) for autonomous penetration testing in an enhanced network simulation environment. Their A3C agents successfully generalized across 72 permutations of three attack scenarios, achieving optimal attack paths and often requiring fewer actions than a decision-tree-based automated penetration testing baseline.

Key insights

  • A3C significantly outperformed Q-learning and DQN, demonstrating robust learning and effective generalization across diverse and permuted network environments, particularly with extended training.
  • Reinforcement learning agents, especially A3C, can achieve optimal or near-optimal attack paths, often requiring fewer actions to compromise systems compared to a deterministic, decision-tree-based baseline.
  • Generalization to unseen network configurations is achievable for A3C agents when trained on a sufficiently diverse subset of environment permutations, indicating potential for adaptable autonomous agents.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.