Reproduce “RealAppliance: Let High-fidelity Appliance Assets Controllable and Workable as Aligned Real Manuals” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuzheng Gao, Yuxing Long, Lei Kang, Yuchong Guo, Ziyan Yu, Shangqing Mao, Jiyao Zhang, Ruihai Wu and 3 more
RealAppliance builds simulated appliances from real product manuals, photos, and measured dimensions, implementing their mechanisms and operating logic. It pairs these assets with manuals in a benchmark of retrieval, part grounding, planning, and recovery from disturbances. On its defined action set, GPT-5 achieves task-completion and task-success rates of 4.30 and 1.22, respectively, showing difficulty on this benchmark, not real-world robot success.
Key insights
- Current multimodal large language models (MLLMs) and embodied planning models demonstrate significant challenges in translating high-level understanding into robust physical manipulation within complex, dynamic, manual-driven environments.
- Failure analysis revealed specific weaknesses in document understanding, accurate part grounding from visual observations, and the ability to adapt to real-time changes in appliance status for closed-loop adjustments.
- The near-zero success rates observed in the full-process inference task highlight the pervasive issue of error accumulation across sub-tasks, indicating that even minor errors can lead to complete task failure in sequential execution.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.