Reproduce “Efficient Two-Stage Gaussian Process Regression Via Automatic Kernel Search and Subsampling” on OpenResearch

About this paper

Shifan Zhao, Jiaying Lu, Ji Yang, Edmond Chow, Yuanzhe Xi

Researchers from Emory University and Georgia Institute of Technology propose a two-stage framework for Gaussian Process Regression to systematically address mean, kernel, and hyperparameter misspecifications. Their approach, combining Kernel Ridge Regression for mean prediction, an automatic kernel search algorithm, and a subsampling-based warm-start strategy for hyperparameter tuning, improves prediction accuracy, enhances uncertainty quantification, and boosts computational efficiency across various real-world datasets, including in safety-critical medical applications.

Key insights

• De-coupling the mean function estimation from uncertainty quantification in GPR allows for more accurate predictions and better calibrated uncertainty estimates.
• An automatic kernel search, guided by a probabilistic misspecification bound, can systematically select an appropriate kernel, enhancing model reliability and uncertainty quantification.
• Optimal hyperparameters derived from subsampled data can serve as effective warm-starts for training on the full dataset, enabling efficient optimization without sacrificing performance.

Summary by alphaXiv.