Reproduce “ReflectDrive-2: Reinforcement-Learning-Aligned Self-Editing for Discrete Diffusion Driving” on OpenResearch

About this paper

Huimin Wang, Yue Wang, Bihao Cui, Pengxiang Li, Ben Lu, Mingqian Wang, Tong Wang, Chuan Tang and 2 more

ReflectDrive-2 lets a driving planner rewrite discrete trajectory tokens after drafting a path from camera views, route instructions, and ego state. The model drafts and edits, while supervised perturbation training and reinforcement learning assign terminal reward to both phases. On NAVSIM, AutoEdit raises PDMS from 89.1 to 91.0 after RL, but results are benchmark-specific and do not establish real-road safety.

Key insights

• Integrating reinforcement learning across both trajectory drafting and self-editing stages is crucial for aligning the entire planning process with closed-loop driving objectives, significantly amplifying the impact of corrections.
• Masked discrete diffusion models provide an intrinsic capability for efficient, in-place token-level revisions, which is particularly effective for correcting structured errors in autonomous driving trajectories.
• Predicting a multi-modal goal-point posterior allows the system to capture diverse behavioral hypotheses, offering flexibility and a pathway to higher performance with improved goal selection.

Summary by alphaXiv.