Reproduce “An end-to-end attention-based approach for learning on graphs” on OpenResearch

About this paper

David Buterez, Jon Paul Janet, Dino Oglic, Pietro Lio

Researchers from the University of Cambridge and AstraZeneca developed Edge-Set Attention (ESA), a simple and efficient end-to-end attention-based model for learning on graph-structured data. This architecture, which processes graphs as sets of edges, achieved state-of-the-art performance across more than 70 diverse graph learning tasks, including molecular property prediction, while demonstrating favorable computational efficiency.

Key insights

• A purely attention-based architecture, operating on edge representations, can achieve competitive or superior performance to existing graph neural networks and graph transformers.
• The strategic interleaving of masked self-attention (incorporating graph connectivity as an inductive bias) and vanilla self-attention allows for effective learning of both local and global graph structures.
• Edge-Set Attention (ESA) demonstrates strong capabilities in transfer learning scenarios, particularly in quantum chemistry tasks, suggesting its utility for foundational models in scientific discovery.

Summary by alphaXiv.