Reproduce “TabularBench: Benchmarking Adversarial Robustness for Tabular Deep Learning in Real-world Use-cases” on OpenResearch

About this paper

Thibault Simonetto, Salah Ghamizi, Maxime Cordy

TabularBench introduces the first comprehensive benchmark for evaluating adversarial robustness in deep learning models on tabular data, addressing the lack of standardized tools by providing a framework, diverse datasets, and over 200 pre-trained robust models. The benchmark reveals that adversarial training combined with data augmentation consistently improves robustness against constrained attacks, with simple techniques like Cutmix often proving highly effective.

Key insights

• High in-distribution performance does not guarantee adversarial robustness, and data augmentations that improve ID performance can sometimes degrade robust performance.
• Evaluating robustness without enforcing domain-specific feature constraints significantly overestimates a model's true resilience, making constrained-aware attacks indispensable.
• Adversarial training consistently improves adversarial accuracy, and while often insufficient alone, it synergistically benefits from data augmentation, with simple techniques like Cutmix often outperforming complex generative models for robust performance.

Summary by alphaXiv.