Reproduce “RoboEgo System Card: An Omnimodal Model with Native Full Duplexity” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiqun Yao, Xiang Li, Xin Jiang, Xuezhi Fang, Naitong Yu, Aixin Sun, Yequan Wang
The Beijing Academy of Artificial Intelligence (BAAI) developed RoboEgo, an omnimodal model system that integrates text, audio, vision, and embodied actions, achieving native full duplexity with a theoretical 80 ms latency. This system enhances real-time human-AI interaction quality and demonstrates advanced embodied action-taking capabilities on physical robots.
Key insights
- Native full-duplex architecture enables drastically lower interaction latency (80 ms theoretical) by parallelizing input processing and simultaneous multi-modal output, surpassing TDM-based methods.
- Efficient stream organization with a "text-first" approach simplifies data annotation, making large-scale multimodal data scaling practical for omnimodal model development.
- Unified architectures can effectively integrate broad omnimodality (text, audio, vision, actions) into a single system, demonstrating capabilities for real-time embodied AI applications.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.