Reproduce “PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations” on OpenResearch

About this paper

Yang Zhang, Jiangyuan Zhao, Chenyou Fan, Fangzheng Yan, Tian Li, Haitong Tang, Sen Fu, Xuan'er Wu and 6 more

PRTS introduces a Vision-Language-Action (VLA) foundation model that integrates reward-label-free Contrastive Reinforcement Learning (CRL) into its pretraining, equipping it with an intrinsic understanding of "goal reachability." This enables the model to quantify the feasibility of achieving language-specified objectives, leading to enhanced performance in long-horizon tasks and zero-shot generalization across diverse robotic platforms.

Key insights

• Language-conditioned Contrastive Reinforcement Learning (LC-CRL) can learn a unified embedding space where the inner product of state-action and goal embeddings approximates the log-discounted goal occupancy, quantifying task progress.
• Implicit, dense goal-reachability supervision can be extracted directly from the temporal structure of offline demonstration trajectories, eliminating the need for explicit reward labels or separate value networks.
• A custom role-aware causal mask allows the VLM backbone to efficiently compute action predictions, state-action representations, and goal representations in a single forward pass, ensuring scalability for large-scale pretraining.

Summary by alphaXiv.