Reproduce “A foundation model of vision, audition, and language for in-silico neuroscience” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Stéphane d'Ascoli, Jérémy Rapin, Yohann Benchetrit, Teon Brooks, Katelyn Begany, Joséphine Raugel, Hubert Banville, Jean-Rémi King
Researchers at FAIR at Meta introduce TRIBE v2, a tri-modal foundation model capable of predicting high-resolution human brain activity (fMRI BOLD signals) across diverse naturalistic conditions and experimental paradigms. The model accurately replicates classical neuroscientific findings in silico and generalizes to new subjects, outperforming linear baselines.
Key insights
- Deep, nonlinear foundation models, like TRIBE v2, significantly outperform linear encoders in modeling complex human brain responses, demonstrating a log-linear increase in accuracy with increasing data volume.
- A single multimodal AI architecture can function as an 'in silico platform' to accurately replicate and predict classic neuroscientific experiments, offering a unified framework for understanding functional brain organization.
- Multisensory integration plays a crucial role in predicting brain activity, with the temporal-parietal-occipital junction and prefrontal cortex showing the largest gains from combining modalities, while different modalities encode complementary brain regions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.