Reproduce “An Optimistic Algorithm for online CMDPS with Anytime Adversarial Constraints” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiahui Zhu, Kihyun Yu, Dabeen Lee, Xin Liu, Honghao Wei
OMDPD learns policies for finite-horizon constrained Markov decision processes with unknown transitions and stochastic or adversarial costs. It estimates the environment and updates a penalty when positive constraint-cost excess occurs, using optimistic mirror descent. With probability at least 1−2δ, it achieves Õ(√K) regret and strong constraint violation, the sum of positive expected-cost excesses per episode, not per-episode safety.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.