Reproduce “Small Language Models for Phishing Website Detection: Cost, Performance, and Privacy Trade-Offs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Georg Goldenits, Philip Koenig, Sebastian Raubitzek, Andreas Ekelhart
Researchers from SBA Research gGmbH and the University of Vienna evaluated the utility of Small Language Models (SLMs) for detecting phishing websites using raw HTML input. Their findings indicate that top-performing SLMs achieved F1-scores up to 0.893, demonstrating a viable, cost-effective, and privacy-preserving alternative to proprietary Large Language Models.
Key insights
- SLMs offer a compelling balance between performance, cost, and data privacy, making them a practical alternative for organizations seeking on-premise phishing detection capabilities without vendor dependency.
- HTML truncation significantly impacts runtime, with 5% truncation proving sufficient to retain critical information for detection while greatly reducing computational load compared to 50% truncation.
- While larger SLMs generally perform better in detection, even mid-sized models like Mistral-Nemo (12b) demonstrate strong F1-scores (0.858), suggesting that high performance can be achieved without the largest parameter counts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.