Reproduce “Conformal Prediction for Zero-Shot Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Julio Silva-Rodríguez, Ismail Ben Ayed, Jose Dolz

Conf-OT introduces an unsupervised, transductive transfer learning framework leveraging optimal transport to provide theoretically guaranteed uncertainty quantification for zero-shot vision-language models. This method enhances the efficiency and adaptiveness of conformal prediction for black-box models while preserving coverage guarantees and improving discriminative accuracy.

Key insights

  • Optimal Transport can effectively bridge domain gaps between foundation model pre-training and downstream tasks in an unsupervised, transductive manner.
  • Improving the efficiency of conformal prediction for black-box models is achievable by adapting the model's representations to the target domain without violating the data exchangeability assumption.
  • The label-marginal distribution estimated from a small calibration set is crucial for guiding optimal transport, enabling effective domain alignment for uncertainty quantification.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.