Reproduce “Mitigating Label Shift in Tabular In-Context Learning via Test-Time Posterior Adjustment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Seunghan Lee
LG AI Research developed DistPFN and DistPFN-T, test-time posterior adjustment methods that mitigate label shift and majority-class bias in tabular in-context learning models like TabPFN. These methods consistently improved accuracy for various TabPFN-based models by up to 13.7% under strong label shifts, while maintaining performance without shifts and enhancing model calibration.
Key insights
- Tabular in-context learning models have a critical, previously unaddressed vulnerability to label shift, manifesting as a 'majority-class bias' that severely impacts performance.
- The explicit conditioning of TabPFN-based models on training data during inference provides a unique opportunity for effective post-hoc, test-time adjustments to mitigate distribution shifts without retraining.
- Approximating the unknown test label distribution with the model's own average predicted distribution or even single-instance predictions proves effective for correcting label shift, offering a practical alternative to complex estimation methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.