Reproduce “TAP-Path: Task-Adaptive Structural and Token Pruning for Efficient and Trustworthy Pathology Foundation Models” on OpenResearch

About this paper

Mehedi Hasan, Ashfak Yeafi, Md Khairul Islam

TAP-Path compresses the 632-million-parameter Virchow2 pathology encoder by retaining task-relevant transformer blocks, pruning image tokens adaptively, and recovering information from multiple depths. On a locked 32-class test set, the resulting model reduces encoder parameters by 24.96% and analytical FLOPs by 35.20% while achieving 87.98% accuracy, 82.38% macro-F1, and strong confidence-based error detection, with external evaluation reaching 91.22% pooled accuracy on two CPTAC cancer cohorts.

Key insights

• Task-adaptive, non-contiguous block selection outperforms simple contiguous truncation at comparable compression levels; token pruning is beneficial when paired with sufficient retained depth but can hurt under more aggressive structural pruning.
• Compression need not be evaluated solely by accuracy: TAP-Path maintains useful probabilistic behavior, including low calibration error, a Brier score of 0.1800, and failure-detection AUROC of 0.9047.
• Rare-class performance is adjustable through the training objective: logit adjustment raises rare-class balanced accuracy to 68.64% but lowers overall accuracy, exposing an explicit majority-versus-minority trade-off.

Summary by alphaXiv.