Reproduce “Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spaces” on OpenResearch

About this paper

Leonard Papenmeier, Luigi Nardi, Matthias Poloczek

Bounce, developed by researchers from Lund University, Stanford University, and Amazon, introduces a High-Dimensional Bayesian Optimization (HDBO) method that reliably optimizes expensive black-box functions across combinatorial, continuous, and mixed input spaces. It leverages nested embeddings and a dynamic trust region strategy, achieving superior performance and efficiency through batch parallelism compared to existing state-of-the-art methods, especially when optimal solutions lack simple structures.

Key insights

• The unreliability of current leading combinatorial BO methods stems from implicit biases in their embedding or sampling strategies towards simple, structurally biased optima, as shown by the performance degradation on randomized benchmarks.
• Nested embeddings can be effectively generalized to robustly handle complex mixed and combinatorial input spaces, allowing Bayesian Optimization to "increase its scope as it learns" while maintaining consistency.
• Intelligent, budget-aware trust region management is critical for both the reliability and efficient batch parallelism of high-dimensional mixed-space Bayesian Optimization.

Summary by alphaXiv.