Reproduce “Towards Expert-level Medical AI for Real-time Video Consultations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mahvish Nagda, Jihyeon Lee, Matthew Thompson, Chunjong Park, Tim Strother, Valentin Liévin, Roma Ruparel, Akshay Goel and 32 more
Google Research and DeepMind developed AMIE (Video), an AI system demonstrating performance comparable to or exceeding human primary care physicians in real-time video consultations. The system achieved a 91% diagnostic accuracy in its top differential diagnosis, outperforming PCPs' 77%, and significantly improved patient communication and perception capabilities over text-based AI.
Key insights
- Integrating real-time audio-visual perception significantly enhances AI's clinical reasoning and information gathering, particularly for physical observation and guided examinations.
- Patient actors strongly prefer video consultations over text for conveying health concerns, ease of use, and feeling understood, highlighting the importance of multimodal communication in healthcare.
- An asynchronous multi-agent architecture enables low-latency dialogue (2.6-second mean turn latency) while allowing for deeper, continuous clinical reasoning and perception.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.