Reproduce “Scalable Autoregressive 3D Molecule Generation” on OpenResearch

About this paper

Austin H. Cheng, Chong Sun, Alán Aspuru-Guzik

Quetzal, a scalable autoregressive model for 3D molecule generation, achieves sample quality competitive with state-of-the-art diffusion models while demonstrating significantly faster generation speeds. It uniquely combines a causal transformer for discrete atom type prediction with a Diffusion Loss-based MLP for continuous 3D coordinate prediction, enabling native support for arbitrary-size generation tasks like hydrogen decoration and scaffold completion.

Key insights

• Autoregressive models can achieve high-quality 3D molecule generation by effectively integrating continuous spatial inductive biases via a Diffusion Loss-based MLP for per-atom coordinate prediction.
• Separating the complexities of quadratic-scaling atom interdependencies (handled by transformers) from localized, per-atom type/position predictions (handled by MLPs) improves efficiency and scalability.
• The model's reliance on specific atom orderings during training and the use of data augmentation (random rotations and translations) are critical for robust performance and generalization to diverse molecular orientations.

Summary by alphaXiv.