Reproduce “Decoding Task Progress from VLA Representations” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Atiksh Bhardwaj, Edward Weiyi Duan, Prithwish Dan, Wei-Chiu Ma, Preston Culbertson

The authors probe π₀.₅’s activations with a linear readout of normalized time remaining in successful demonstrations, then use stalled progress estimates to flag out-of-distribution rollouts without failure examples. In simulation, the detector reaches 0.960 ± 0.013 episode-level AUROC on held-out perturbation modes, versus SAFE-MLP’s 0.894 ± 0.063; on held-out tasks it scores 0.851 versus 0.917. Evidence is limited to the π₀.x model family and simulation.

Key insights

  • Task progress is linearly decodable from VLA internal representations, and this feature primarily emerges during large-scale robotic pre-training rather than task-specific fine-tuning.
  • While the task progress feature is observable (decodable), it was not found to be linearly steerable via simple interventions on activations, indicating a distinction between representation and causal control.
  • Counterfactual training, incorporating language perturbations, can enhance the progress probe's sensitivity to linguistic changes, providing a more robust signal for task understanding and diagnosis.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.