Reproduce “PlanTRansformer: Unified Prediction and Planning with Goal-conditioned Transformer” on OpenResearch

About this paper

Constantin Selzer, Fabina B. Flohr

PlanTRansformer (PTR) unifies trajectory prediction and planning in autonomous driving by extending the Motion Transformer to incorporate planning constraints like dynamic feasibility and collision avoidance. This goal-conditioned framework generates actionable and safer trajectories for both ego and surrounding agents. On the Waymo Open Motion Dataset, PTR achieves a 4.3% improvement in average mAP for prediction and reduces the collision rate to 2.16% in open-loop planning, outperforming GameFormer.

Key insights

• Directly integrating planning constraints (differentiable collision and dynamics losses) and map-based features (reachable lanes) into the prediction model significantly improves both the safety and actionability of generated trajectories.
• Initializing the decoder's query content features with high-level command embeddings effectively incorporates semantic prior knowledge, leading to more intention-aligned and accurate multimodal predictions.
• A teacher-student training strategy, where ground-truth commands for surrounding agents are progressively masked, enables the model to generalize and generate plausible trajectories even when surrounding agent intentions are unknown at inference time.

Summary by alphaXiv.