Reproduce “XxaCT-NN: Structure Agnostic Multimodal Learning for Materials Science” on OpenResearch

About this paper

Jithendaraa Subramanian, Linda Hung, Daniel Schweigert, Santosh Suram, Weike Ye

The Toyota Research Institute researchers developed XxaCT-NN, a multimodal deep learning framework that accurately predicts material properties from experimentally accessible data like elemental composition and X-ray diffraction (XRD) patterns, circumventing the need for explicit atomic structure. The model demonstrated strong performance on a 5-million-sample dataset, achieving a 28.2 meV/atom MAE for formation energy prediction, approaching the performance of structure-based models while significantly outperforming prior structure-agnostic methods.

Key insights

• Multimodal models, particularly XxaCT-NN's cross-attention fusion, significantly outperform unimodal approaches and prior bimodal methods that use simpler fusion, demonstrating the power of integrating diverse data.
• Self-supervised pretraining objectives like Masked XRD Modeling (MXM) and contrastive learning effectively leverage unlabeled data, accelerate model convergence by up to 4.2 times, and improve downstream prediction accuracy and representation quality.
• Multimodal models exhibit stronger scaling laws with larger datasets compared to unimodal models, yielding amplified performance gains as data volume increases and suggesting that the field has not yet reached a performance plateau.

Summary by alphaXiv.