Reproduce “DA-MoE: Addressing Depth-Sensitivity in Graph-Level Analysis through Mixture of Experts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zelin Yao, Chuang Liu, Xianke Meng, Yibing Zhan, Jia Wu, Shirui Pan, Wenbin Hu
The DA-MoE framework enables Graph Neural Networks (GNNs) to adapt their aggregation depth based on the scale of individual input graphs, addressing the challenge of depth-sensitivity. This method consistently improved performance across various graph-level, node-level, and link-level tasks on datasets like TU and OGB, for instance, boosting ROC-AUC by 2.75% on "ogbg-molhiv" and HITS@100 by 14.94% on "ogbl-ppa" when integrated with GIN or GCN backbones.
Key insights
- GNNs exhibit a measurable "depth-sensitivity," where optimal aggregation depth for learning representations directly correlates with the scale of the input graph.
- An adaptive approach using a Mixture of Experts, where experts are GNNs of varying depths, allows dynamic selection of appropriate aggregation scales based on graph characteristics.
- Leveraging graph structural information within the gating network significantly enhances the accuracy of expert selection compared to simpler, non-graph-aware gating mechanisms.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.