Reproduce “Simulation-aided Reinforcement Learning with Control Variates” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Omer Hemo, Ido Sefi, Mirco Mutti, Aviv Tamar, Kfir Y. Levy
The authors introduce CV-Qmix, using simulated experience as a control variate to reduce noise in real-data policy-value estimates rather than treating simulation as the target. Its gradient estimator is unbiased under stated density-ratio and sampling assumptions. In MuJoCo Humanoid evaluation, it beats real-only estimation at every real-data budget and ordinary mixing beyond roughly 2,000 samples; this is benchmark evidence, not physical-robot validation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.