Reproduce “Critic Regularized Regression” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ziyu Wang, Alexander Novikov, Konrad Zolna, Jost Tobias Springenberg, Scott Reed, Bobak Shahriari, Noah Siegel, Josh Merel and 3 more

CRR is an offline reinforcement-learning method for learning policies from fixed datasets. It trains a critic to compare recorded actions with actions proposed by the current policy, then filters or weights recorded actions during policy learning. On simulated Humanoid Corridor, its exponential-weighting variant scored 918 ± 14, versus ABM’s 64 ± 3. It does not establish universal or real-world superiority.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.