Reproduce “Metropolis-Hastings Dominates Importance Resampling for Policy Composition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alexey Kurennoy, Ramil Yarullin, Fergal Reid
The authors introduce a decoding-time method that combines specialist policies and uses recorded generation weights to correct local next-token mixing toward the weighted product distribution over complete responses. At matched rollout budgets, independence Metropolis–Hastings is no farther from this target than sampling-importance-resampling under any convex f-divergence. On restricted continuation trees, two-candidate Metropolis–Hastings reduces mean KL divergence by 96.8%, versus 76.0% for resampling.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.