Reproduce “EmBARDiment: an Embodied AI Agent for Productivity in XR” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Riccardo Bovo, Steven Abreu, Karan Ahuja, Eric J Gonzalez, Li-Te Cheng, Mar Gonzalez-Franco

An embodied AI agent framework named EmBARDiment, developed by researchers from Google, Imperial College London, and the University of Groningen, enhances productivity in multi-window Extended Reality (XR) environments by integrating gaze-driven contextual memory with Large Language Models. User studies demonstrated that this approach reduced interaction attempts and increased user satisfaction compared to baseline and full-context methods.

Key insights

  • Implicitly deriving context from user eye-gaze within an XR environment significantly enhances the efficiency and user satisfaction of LLM-powered agents.
  • Establishing a "shared theory of mind" where the AI agent implicitly understands the user's focus is more effective than relying solely on explicit prompts, reducing cognitive load.
  • Providing selectively curated, attention-based contextual information to LLMs outperforms providing a broad, unfiltered context from all open windows.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.