Reproduce “Chem3DLLM: 3D Multimodal Large Language Models for Chemistry” on OpenResearch

About this paper

Lei Jiang, Shuzhou Sun, Biqing Qi, Yuchen Fu, Xiaohua Xu, Yuqiang Li, Dongzhan Zhou, Tianfan Fu

Researchers from the University of Science and Technology of China and Shanghai AI Laboratory developed Chem3DLLM, a protein-conditioned multimodal large language model designed to process and generate 3D molecular structures. The model integrates 3D chemical information into LLMs through a reversible tokenization scheme and refines generation using scientific feedback, achieving state-of-the-art performance in molecular conformation generation (100% validity) and structure-based drug design (average Vina score of -7.21).

Key insights

• 3D molecular structures can be effectively tokenized into a discrete, compact, and fully reversible format for LLMs without losing critical geometric information, enabling native 3D reasoning within these models.
• Jointly training a multimodal LLM on both molecular conformation generation and structure-based drug design tasks leads to synergistic performance improvements, enhancing ligand binding affinity and chemical validity.
• Integrating scientific prior knowledge via reinforcement learning with stability-centric reward functions is crucial for guiding LLM generation towards chemically plausible and thermodynamically stable molecular structures.

Summary by alphaXiv.