Reproduce “High-Dimensional Gaussian Process Regression with Soft Kernel Interpolation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chris Camaño, Daniel Huang
SoftKI provides a method for high-dimensional Gaussian Process regression that scales to large datasets by introducing data-adaptive interpolation points via a novel softmax interpolation scheme. The approach, from Caltech and Base26, effectively manages computational complexity and numerical stability, achieving strong performance on high-dimensional molecular datasets like MD22, outperforming previous methods and being significantly faster than exact GPs.
Key insights
- It is possible to overcome the dimensionality dependency of traditional Structured Kernel Interpolation by using learned, data-adaptive interpolation points combined with a softmax weighting scheme.
- A softmax-based interpolation allows for flexible, effectively sparse kernel approximations where each data point's influence from interpolation points decays exponentially with distance.
- Numerical stability in MLL optimization and posterior inference for large-scale GPs can be achieved through a combination of approximate MLL (Hutchinson's pseudoloss) as a fallback and robust linear solvers (QR-stabilized).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.