Reproduce “Weighted Linear Bandits for Non-Stationary Environments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yoan Russac, Claire Vernade, Olivier Cappé
Researchers at CNRS, Inria, ENS, Université PSL, and DeepMind developed D-LinUCB, an algorithm for stochastic linear bandits that achieves asymptotically optimal dynamic regret rates of O(d^(2/3)B_T^(1/3)T^(2/3)) in non-stationary environments. This method utilizes exponentially discounted weighted least-squares, is recursively implementable, and adapts effectively to both abrupt and slowly-varying parameter changes.
Key insights
- Novel deviation inequalities were derived for sequential weighted regularized least-squares, providing a theoretical foundation for establishing confidence bounds in dynamic settings.
- Exponential discounting in weighted least-squares allows for a smooth and robust mechanism to
- forget
- outdated information, enabling effective adaptation to both abrupt and slowly-varying environmental changes.
- By carefully tuning the discount factor "gamma" as a function of the total variation budget (B_T) and time horizon (T), asymptotically optimal dynamic regret rates can be achieved.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.