Reproduce “World Models Need Not Be Right to Rank Policies Right: Relative Bounds and Audit Limits” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ahanaf Hasan Ariq, Hasaan Ahmad
The authors analyze how learned world models can rank reinforcement-learning policies despite errors, and introduce RAVEL, which uses model scores to order candidates but requires held-out real audits to eliminate them. At horizon 500, ranking agreement reaches Kendall’s τ of 0.962 for cheetah-run and 0.800 for walker-walk, but paired-audit correlations are near zero; results cover two state-based tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.