Reproduce “Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and MLPs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chenxiao Yang, Qitian Wu, Jiahua Wang, Junchi Yan
Graph Neural Networks (GNNs) derive their performance advantage over Multi-Layer Perceptrons (MLPs) largely from an intrinsic generalization capability, rather than solely superior representational power. Researchers from Shanghai Jiao Tong University introduced Propagational MLPs (PMLPs), which train as MLPs but apply GNN-like message passing at inference, achieving comparable performance to full GNNs with significantly faster training.
Key insights
- A major source of GNNs' performance gain is an intrinsic generalization capability inherent in their architecture, particularly during inference, rather than solely enhanced representational power learned during training.
- The architectural change to include message passing at test time (as in PMLP) transforms the model's feature map, leading to superior generalization without altering learned weights or the training process.
- GNNs and PMLPs exhibit a smoother, degree- and feature-dependent transition to linear behavior for out-of-distribution samples compared to MLPs, indicating better tolerance for OoD samples near the training data range.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.