Reproduce “The Promises and Pitfalls of Deep Kernel Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sebastian W. Ober, Carl E. Rasmussen, Mark van der Wilk

The authors examine whether deep kernel learning (DKL), which applies a Gaussian process (GP) to neural-network features, avoids overfitting while estimating uncertainty. In a toy regression, exact full-batch DKL overfits more than a same-depth neural network. Minibatching mitigates this in reported comparisons; on CIFAR-10, larger batches worsen fine-tuned SVDKL’s log likelihood despite unchanged accuracy, within these settings.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.