Reproduce “ANCHOR: KL-Anchored Adaptation for Stable Offline-to-Online RL” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Geonwoo Cho, Jaegyun Im, Doyoon Kim, Lexin Li
ANCHOR addresses online adaptation of pretrained robot-control policies when offline data are unavailable. It penalizes divergence from a fixed pretrained policy, then linearly reduces that penalty to zero. Across six tasks, its mean endpoint success rate was 0.957 versus 0.842 for WSRL, and early-failure score was 0.117 versus 0.367; benefits depend on useful pretrained behavior and do not establish convergence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.