Reproduce “A Tutorial on Thompson Sampling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Daniel Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, Zheng Wen
Thompson sampling learns through consequential choices by sampling a plausible model and selecting its best action, supporting current rewards and future learning. In a simulated 20-stage binomial-bridge shortest-path task with 184,756 routes, per-period regret fell toward zero and the travel-time-to-optimal ratio approached 1 faster than tested ε-greedy methods. This result is limited to this simulation.
Key insights
- Thompson sampling inherently balances exploration and exploitation by probabilistically favoring actions with higher posterior probability of being optimal.
- The algorithm generalizes effectively to problems with complex information structures and large action spaces by modeling dependencies and utilizing prior knowledge.
- Theoretical analyses demonstrate TS's asymptotic optimality for classical bandits and strong regret bounds for more general online decision problems, linked to concepts like eluder dimension.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.