Reproduce “Real-time Reconstruction of Human Visual Perception from fMRI” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rishab S. Iyer, Jiaxin Cindy Tu, Cesar Kadir Torrico Villanueva, Anish Mahishi, Ross P. Kempner, Jacob S. Prince, Ernest W. Lo, Akash Bhowmick and 5 more
A Princeton University-led study adapted a state-of-the-art fMRI decoding pipeline, MindEye2, for real-time operation using a 3T MRI, enabling the first reported real-time reconstruction of human visual perception at single-trial resolution. The system achieved image retrieval within 9.5 seconds and reconstruction within 14.5 seconds from stimulus onset, with 'fast' real-time achieving 36% image retrieval accuracy for a pool of 50 images.
Key insights
- State-of-the-art, fine-grained decoding of visual perception from fMRI can be achieved in real-time on widely available 3T MRI scanners with minimal new participant training data (approximately one hour).
- A trade-off exists between real-time processing speed (stimulus delay) and decoding performance, with longer delays leading to improved BOLD response estimation and better accuracy, showing diminishing returns after approximately 30 seconds of delay.
- Pretraining on extensive 7T fMRI datasets significantly enhances real-time decoding performance, but the system can still achieve above-chance results with limited 3T data alone.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.