Reproduce “PRISM: Exploring Heterogeneous Pretrained EEG Foundation Model Transfer to Clinical Differential Diagnosis” on OpenResearch

About this paper

Jeet Bandhu Lahiri, Parshva Runwal, Arvasu Kulkarni, Mahir Jain, Aditya Ray Mishra, Siddharth Panwar, Sandeep Singh

This research explores the impact of population heterogeneity in pretraining data on EEG foundation models, demonstrating that multi-source pretraining improves performance on a novel and challenging clinical differential diagnosis task. The work also dissects inconsistencies between established evaluation benchmarks, revealing significant ranking reversals due to methodological variations.

Key insights

• Population heterogeneity in pretraining data (D2) leads to more robust and adaptable EEG representations under fine-tuning, outperforming narrow-source data (D1) on most benchmark tasks and significantly on a novel clinical diagnosis task.
• The choice of classification head and fine-tuning strategy significantly impacts downstream performance, indicating that optimal adaptation depends on the specific task.
• Evaluation protocol inconsistencies (e.g., train/validation splits, checkpoint selection) cause substantial performance discrepancies and model ranking reversals, highlighting an urgent need for standardization in the field.
• Targeted data diversity can be more impactful than indiscriminate data scale, as PRISM (three source corpora) matched or exceeded models trained on significantly larger, less diverse datasets.

Summary by alphaXiv.