Reproduce “Efficient kernelized bandit algorithms via exploration distributions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bingshan Hu, Zheng He, Danica J. Sutherland
The paper introduces a kernelized bandit that randomizes a scalar uncertainty bonus when selecting arms. Each round, it adds a sampled weight times the uncertainty estimate to the reward estimate and chooses the arm with the highest score. Under stated conditions, bounded control quantities yield expected regret scaling as gamma_T times square root T, up to logarithmic factors. Simple-Bernoulli performs best on four synthetic tasks, but the results establish neither optimal regret nor a universal advantage.
Key insights
- Upper Confidence Bound (UCB) and Thompson Sampling (TS) approaches for kernelized bandits can be unified under a generic framework centered on "exploration distributions."
- Utilizing a scalar random variable w_t from tailored exploration distributions effectively manages the exploration-exploitation trade-off and simplifies algorithm design.
- Controlled randomization, facilitated by these exploration distributions, can lead to enhanced practical performance without compromising theoretical state-of-the-art regret bounds.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.