Reproduce “Scalable Gaussian Processes on Discrete Domains” on OpenResearch

About this paper

Vincent Fortuin, Gideon Dresdner, Heiko Strathmann, Gunnar Rätsch

The paper develops a sparse Gaussian process for discrete strings by selecting inducing strings with simulated annealing, including candidates outside the training set, and compares greedy selection and determinantal point process sampling. On a reported UCI DNA-splicing subset, it achieves AUPRC 0.676 ± 0.002 versus 0.678 ± 0.002 for the full GP, with faster inference, but not better calibration.

Key insights

• Simulated Annealing (SA) consistently yields superior performance for inducing point selection in sparse GPs on discrete domains, largely due to its ability to explore points beyond the training set.
• Intelligent selection of inducing points, especially with methods like SA, allows sparse GPs to closely approximate the predictive performance and uncertainty calibration of full GPs.
• The distribution of optimally chosen inducing points adapts to the task, clustering near decision boundaries in classification and spreading evenly in regression.

Summary by alphaXiv.