Reproduce “Limited Linguistic Diversity in Embodied AI Datasets” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Selma Wanna, Agnes Luhtaru, Jonathan Salfity, Ryan Barron, Juston Moore, Cynthia Matuszek, Mitch Pryor

Researchers at Los Alamos National Laboratory and collaborating institutions conducted a systematic audit of language instructions in widely used Vision-Language-Action (VLA) datasets, revealing pervasive linguistic uniformity characterized by high command duplication, restricted vocabulary, and a lack of complex syntactic structures. This comprehensive analysis provides a foundation for understanding current VLA model generalization gaps and guides strategies for creating more linguistically diverse datasets.

Key insights

  • Existing VLA datasets show pervasive linguistic uniformity with high command duplication and limited vocabulary, which likely contributes to observed generalization failures in VLA models.
  • Semantic analysis reveals low intrinsic dimensionality and constrained verb-direct object co-occurrence within VLA datasets, indicating narrow and repetitive coverage of actions and objects.
  • VLA datasets exhibit a strong bias towards simple, procedural, linear instructions, almost entirely lacking complex linguistic structures crucial for robust reasoning, such as negation or conditionals.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.