Reproduce “Detecting Concept Drift in Evolving Malware Families Using Rule-Based Classifier Representations” on OpenResearch

About this paper

Tomáš Kalný, Martin Jureček, Mark Stamp

Tomáš Kalný, Martin Jureček, and Mark Stamp introduced a structural method for identifying concept drift in malware classifiers by tracking changes in decision tree rulesets. Their work revealed that shifts in feature importance within these rules, especially when classifying between malware families, correlated strongly (Spearman ρ up to 0.8) with subsequent model performance degradation over 2-month periods.

Key insights

• Changes in interpretable decision tree rulesets, specifically feature importance, serve as meaningful indicators of concept drift and correlate with classifier performance degradation and data distribution shifts.
• Concept drift signals are more clearly detectable and robust in "family-versus-family" classification scenarios compared to "family-versus-benign," as it highlights changes in family-specific characteristics.
• Fixed temporal windows (e.g., 2 months) were more effective for observing meaningful concept drift signals than clustering-based windowing, which often suffered from small sample sizes and inconsistent results.

Summary by alphaXiv.