Reproduce “Phishing Detection in the Gen-AI Era: Quantized LLMs vs Classical Models” on OpenResearch

About this paper

Jikesh Thapa, Gurrehmat Chahal, Serban Voinea Gabreanu, Yazan Otoum

Researchers at Algoma University compared traditional machine learning, deep learning, and quantized small Large Language Models (LLMs) for phishing detection against AI-generated threats. The study found that while deep learning models achieved the highest accuracy at 98.77%, classical ML offered superior speed, and quantized LLMs provided interpretable explanations and adversarial resilience, despite lower accuracy up to 79%.

Key insights

• Quantized small LLMs offer interpretable explanations for their classifications and demonstrate potential resilience against LLM-rephrased adversarial content, providing a unique advantage over black-box models.
• Deep Learning models, particularly the Bi-Directional GRU with Leaky ReLU, achieve very high accuracy (98.77%) and strong discriminatory power with efficient inference times, making them suitable for bulk email screening.
• Traditional ML models provide significantly faster inference (e.g., Logistic Regression is 1.38 million times faster than LLMs) with competitive accuracy, indicating their viability for real-time, high-throughput phishing detection.

Summary by alphaXiv.