Reproduce “Correcting Class Imbalance in Prior-Data Fitted Networks for Tabular Classification” on OpenResearch

About this paper

Samuel McDowell, Nathan Stromberg, Lalitha Sankar

This research investigates class imbalance correction for Prior-data Fitted Networks (PFNs), demonstrating that simple decision-level thresholding and data-level downsampling effectively improve minority class accuracy. The study reveals TabPFNs exhibit a specific majority-class bias, guiding the successful application of thresholding to achieve maximal balanced accuracy.

Key insights

• Prior-data Fitted Networks (PFNs) exhibit a consistent majority-class bias when operating on imbalanced datasets, rather than general over- or under-confidence.
• Increasing class imbalance significantly reduces the Area Under the Curve (AUC) in ROC analysis, but adjusting the decision threshold can effectively shift the operating point towards optimal balanced accuracy.
• Simple, classical class imbalance correction methods (thresholding, downsampling) can be highly effective for in-context learning models like PFNs without complex retraining or weight modifications.

Summary by alphaXiv.