Reproduce “Local Latent Space Bayesian Optimization over Structured Inputs” on OpenResearch

About this paper

Natalie Maus, Haydn T. Jones, Juston S. Moore, Matt J. Kusner, John Bradshaw, Jacob R. Gardner

LOL-BO is a method for optimizing structured inputs, such as molecules or arithmetic expressions, which combines local Bayesian optimization with an adaptive latent space by jointly training a deep autoencoder and a Gaussian Process surrogate model. This approach demonstrates up to 20 times better objective values on benchmarks like molecular penalized Log P and improves sample efficiency for expensive tasks like DRD3 receptor docking affinity.

Key insights

• Optimizing structured inputs in latent spaces benefits significantly from a co-design approach where the deep autoencoder and the Gaussian Process surrogate are jointly inferred, enabling the latent space to adapt.
• Dynamically shaping the latent space to align with the smoothness assumptions of Gaussian Processes and trust region methods is critical for robust and efficient exploration.
• Employing advanced deep autoencoder architectures, such as transformer-based VAEs with the SELFIES molecular representation, leads to more stable and effective latent spaces suitable for optimization.

Summary by alphaXiv.