Reproduce “Probing a Vision-Language-Action Model for Symbolic States and Integration into a Cognitive Architecture” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hong Lu, Hengxu Li, Prithviraj Singh Shahani, Stephanie Herbers, Matthias Scheutz
This research from Tufts University demonstrates that a Vision-Language-Action (VLA) model, OpenVLA, encodes high-level symbolic representations of object properties, relations, and action states within its hidden layers, which linear probes can decode with high accuracy. The work integrates these extracted symbolic states into the DIARC cognitive architecture, enabling real-time monitoring and paving the way for more interpretable and robust robotic systems.
Key insights
- High-level symbolic information, such as object relations and action subgoals, is robustly encoded within the internal representations of VLA models like OpenVLA.
- Linear probing is an effective method for extracting these implicit symbolic states from VLA hidden layers for robotic applications.
- Integrating extracted symbolic states from VLAs into cognitive architectures offers a pathway to combine the generalization capabilities of deep learning with the interpretability and reasoning of symbolic AI.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.