Reproduce “TSTEM: A Cognitive Platform for Collecting Cyber Threat Intelligence in the Wild” on OpenResearch

About this paper

Prasasthy Balasubramanian, Sadaf Nazari, Danial Khosh Kholgh, Alireza Mahmoodi, Justin Seby, Panos Kostakos

The TSTEM platform, developed by the Center for Ubiquitous Computing at the University of Oulu, introduces an open-source, cognitive framework for real-time collection and extraction of cyber threat intelligence from diverse online sources, including Twitter, clear web, and dark web. It leverages state-of-the-art AI/NLP models like fine-tuned BERT and Longformer, achieving up to 98% accuracy in classification and extracting verifiable, novel Indicators of Compromise through an autonomously deployed, scalable microservice architecture.

Key insights

• State-of-the-art transformer-based AI/NLP models like BERT and Longformer achieve high accuracy in classifying relevant CTI and extracting Indicators of Compromise (IOCs) from diverse text sources.
• Combining focused web crawling with real-time streaming architectures and open-source principles enables transparent and efficient collection of CTI "in the wild."
• Infrastructure as Code (IaC) and MLOps practices are crucial for autonomously deploying and managing scalable, reliable, and maintainable CTI collection platforms.

Summary by alphaXiv.