Reproduce “Information-Directed Offline-to-Online Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Keru Chen

The paper develops IDS for online exploration after offline learning, selecting actions by balancing expected regret against information about uncertainty remaining after conditioning on offline data. In a hidden-mode experiment, vanilla IDS achieves cumulative regret 0.15 versus 3.31 for greedy and UCB and 1.79 for Thompson sampling over 500 steps. This result depends on an informative, sufficiently cheap probe.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.