Reproduce “An Exponential Lower Bound for Linearly-Realizable MDPs with Constant Suboptimality Gap” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuanhao Wang, Ruosong Wang, Sham M. Kakade
The authors show online learning can require exponentially many samples despite exact linear realizability of the action-value function and a constant action-value gap. Shared features hide the best of exponentially many actions, while transitions may end episodes before evidence appears. In the worst case, achieving 0.05 expected-return accuracy requires exponentially many interaction samples in the minimum of feature dimension and horizon.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.