Reproduce “Drifting Field Policy: A One-Step Generative Policy via Wasserstein Gradient Flow” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Juil Koo, Mingue Park, Jiwon Choi, Yunhong Min, Minhyuk Sung
DFP maps noise and state directly to actions, then updates its action distribution using critic-ranked top-K candidates and replay data. It generates each candidate in one forward pass, without an ODE trajectory. Across 12 simulated manipulation tasks, DFP averages 95.8% success, ranking first on nine and second on three, versus 80.3% for MVP. These results concern simulated continuous-action tasks.
Key insights
- Framing policy updates as a reverse-KL Wasserstein-2 gradient flow provides a principled learning rule, allowing direct descent in probability space at the action output level.
- The Wasserstein gradient structurally decomposes into components driving towards higher Q-values and maintaining a trust region with respect to an anchor policy, balancing exploitation and exploration.
- The non-ODE drifting backbone exhibits a unique synergy with the top-K critic-selected supervision, significantly enhancing performance compared to applying the same supervision to ODE-based architectures due to direct signal propagation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.