Reproduce “PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
DeepCybo Team, Yu Bin, Haipeng Cao, Zheng Chang, Kai Chen, Youning Chen, Kailin Deng, Yichao Du and 46 more
PhysBrain 1.5 combines embodied scene understanding, end-effector trajectory prediction, and future RGB, depth, and robot-mask prediction in one Qwen3-VL-Instruct-based 8B model. It uses shared autoregressive tokens and trains on human, robot, and simulated experience. Across 28 benchmarks, it scores 72.5 versus 59.5 for the base model, but this is not robot success or closed-loop autonomy.
Key insights
- Human interaction videos can supply large-scale supervision for object interaction, affordances, task progress, motion, and state transitions before robot-specific fine-tuning.
- A shared discrete-token interface lets semantic reasoning, trajectory continuation, and future-state forecasting exchange representations within a single autoregressive model.
- The approach improves embodied benchmark performance without fully discarding general multimodal competence, but it leaves coordinate-frame variation, pseudo-label quality, and real-world control reliability as open issues.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.