Reproduce “A Large Encoder-Decoder Family of Foundation Models For Chemical Language” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Eduardo Soares, Victor Shirasuna, Emilio Vital Brazil, Renato Cerqueira, Dmitry Zubarev, Kristin Schmidt
IBM Research introduces SMI-TED289M, a family of large encoder-decoder chemical foundation models pre-trained on 91 million curated SMILES strings, achieving state-of-the-art performance across 11 MoleculeNet benchmarks, including an average MAE of 1.3246 on QM9, and demonstrating compositional molecular representations with a Tanimoto score of 0.52 for few-shot learning tasks.
Key insights
- High-quality, curated pre-training data, specifically 91 million unique and valid SMILES strings, is crucial for developing robust chemical foundation models.
- Fine-tuning the pre-trained encoder-decoder model significantly improves performance, enabling state-of-the-art results across complex classification and regression molecular property prediction tasks.
- The learned latent space exhibits compositionality, separating molecular families and respecting hierarchical distances, which allows for few-shot learning and implicitly captures group contribution-like principles crucial for chemical reasoning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.