Reproduce “Next-Generation Phishing: How LLM Agents Empower Cyber Attackers” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Khalifa Afane, Wenqi Wei, Ying Mao, Junaid Farooq, Juntao Chen

Researchers from Fordham University and the University of Michigan-Dearborn analyzed how Large Language Models can create sophisticated phishing emails with higher evasion rates while also exploring their potential for strengthening detection systems. The study found that LLM-rephrased emails achieved a 16.4% evasion rate, but training detection models on LLM-augmented datasets significantly improved their ability to detect these advanced threats.

Key insights

  • LLM-rephrased phishing emails significantly narrow the decision boundary between legitimate and malicious content, making them harder for detection systems to classify.
  • Current traditional, machine learning, and even LLM-based phishing detection systems experience performance degradation when faced with LLM-enhanced phishing emails.
  • Training machine learning models on LLM-augmented datasets, containing diverse variations of phishing emails, substantially improves their ability to detect sophisticated LLM-rephrased attacks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.