Reproduce “Sari Sandbox: A Virtual Retail Store Environment for Embodied AI Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Janika Deborah Gajo, Gerarld Paul Merales, Jerome Escarcha, Brenden Ashley Molina, Gian Nartea, Emmanuel G. Maminta, Juan Carlos Roldan, Rowel O. Atienza
Researchers at the University of the Philippines, Diliman, developed "Sari Sandbox," a high-fidelity virtual retail store environment designed for benchmarking embodied AI agents against human performance in shopping tasks. The platform, featuring 250+ diverse products and a Python API, also includes SariBench, a dataset of human demonstrations, revealing that current VLM-powered agents are significantly less efficient than humans in these tasks.
Key insights
- High-fidelity simulation environments like Sari Sandbox require significant optimization efforts, utilizing techniques such as Level of Detail (LOD) and frustum/occlusion culling, to achieve acceptable performance (23-35 FPS) under demanding graphical and physics loads.
- Accurately processing and understanding complex visual information, such as reading rotated or stylized text on product labels, remains a challenge for current OCR systems, highlighting a critical area for agent perception improvement.
- Current VLM-powered embodied agents, despite employing frameworks like ReAct and explicit memory modeling, exhibit substantial computational overhead and reasoning limitations, resulting in significantly lower efficiency and success rates compared to human performance in retail tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.