Reproduce “SADA: Safe and Adaptive Aggregation of Multiple Black-Box Predictions in Semi-Supervised Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiawei Shan, Zhifeng Chen, Yiming Dong, Yazhen Wang, Jiwei Zhao

SADA combines multiple black-box predictions with labeled and unlabeled data for inference and prediction when labeled outcomes are scarce. It learns weights from labeled examples and adds a correction with zero expectation under the same-distribution assumption, without requiring accurate predictions. Its guarantees are asymptotic or rate-based; in simulation, standard deviation was 0.727 versus 0.799 and 0.801 for single-prediction PPI++.

Key insights

  • The proposed augmented estimating equation and loss function remain unbiased, guaranteeing that SADA never performs worse than labeled-only methods, regardless of black-box prediction quality.
  • SADA's data-driven weighting mechanism automatically adapts to the quality of available predictions, achieving faster convergence rates or semiparametric efficiency bounds if highly accurate predictions exist.
  • A unified framework can effectively address both statistical inference for parameters and improved prediction accuracy by formulating optimal weight derivation, catering to distinct scientific and industrial goals.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.