Reproduce “GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Han Zhong, Wei Xiong, Sirui Zheng, Liwei Wang, Zhaoran Wang, Zhuoran Yang, Tong Zhang
Researchers from Peking University, The Hong Kong University of Science and Technology, Northwestern University, and Yale University introduce the Generalized Eluder Coefficient (GEC) as a unified complexity measure and the Generic Posterior Sampling Algorithm for Interactive Decision Making (GPS-IDM) for sample-efficient online reinforcement learning. This framework encompasses MDPs, POMDPs, and PSRs, demonstrating that GEC upper-bounds existing complexity measures and yields provable sublinear regret bounds for the generic algorithm.
Key insights
- The GEC unifies numerous existing complexity measures, such as Bellman Eluder Dimension, Bilinear Class, and Witness Rank, by demonstrating that it can upper-bound them across diverse RL problems.
- Introduced Generalized Regular PSRs as a new tractable class of partially observable models that encompasses several known POMDP classes (e.g., Weakly Revealing, Latent, Decodable, Low-rank POMDPs).
- Developed new $ extrm{ extless}i extrm{ extgreater} extrm{l} extrm{ extunderscore} extrm{2} extrm{ extless}/ extrm{i} extrm{ extgreater} $ Eluder techniques for analyzing posterior sampling, which were crucial for establishing sharper regret bounds in partially observable environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.