Reproduce “Safe Linear Stochastic Bandits” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Kia Khezeli, Eilyan Bitar

Researchers from Cornell University introduced the safe linear stochastic bandit framework, which incorporates explicit stagewise safety constraints into linear stochastic bandit problems. Their Safe Exploration and Greedy Exploitation (SEGE) algorithm ensures that the expected reward of chosen actions remains above a predefined safety threshold with high probability at every stage, while simultaneously achieving an O(rootT log T) expected regret bound, which is near-optimal.

Key insights

  • Stagewise safety can be rigorously guaranteed in linear bandits by constructing actions (both exploratory and exploitative) whose lower confidence bounds on expected rewards meet a predefined threshold.
  • Balancing aggressive exploration with exploitation under safety constraints is achievable by dynamically selecting between a carefully constructed safe exploration arm and a conditionally safe greedy arm.
  • Maintaining a near-optimal regret rate, O(rootT log T), is possible even with stringent stagewise safety requirements, demonstrating that strong learning efficiency does not need to be sacrificed for real-time safety.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.