Reproduce “PHemoNet: A Multimodal Network for Physiological Signals” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Eleonora Lopez, Aurelio Uncini, Danilo Comminiello
PHemoNet, developed by researchers at Sapienza University of Rome, introduces a fully hypercomplex multimodal neural network that processes minimally preprocessed EEG, ECG, GSR, and eye-tracking data to classify emotional states along arousal and valence dimensions. The system achieved F1-scores of 0.401 for arousal and 0.505 for valence classification on the MAHNOB-HCI dataset, demonstrating performance gains over previous state-of-the-art methods.
Key insights
- Processing physiological signals in their "natural domain" using parameterized hypercomplex multiplication (PHM) layers allows a neural network to learn optimal representations directly from minimally processed data.
- Hypercomplex neural networks can model complex relationships and correlations more efficiently and with fewer parameters compared to traditional real-valued networks, leading to better generalization.
- A hierarchical architecture with modality-specific hypercomplex encoders and a unified hypercomplex fusion module effectively integrates diverse physiological signals while adapting to their inherent dimensionalities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.