Reproduce “Flexible Infinite-Width Graph Convolutional Neural Networks” on OpenResearch

About this paper

Ben Anson, Edward Milsom, Laurence Aitchison

The authors introduce a graph convolutional deep kernel machine (GCDKM) that learns task-specific node similarities while retaining an infinite-width kernel framework. A regularization parameter ν controls flexibility, with the fixed-kernel GCNNGP recovered as ν tends to infinity. On low-homophily node-classification tasks, flexibility tends to help. On Roman Empire, GCDKM scores 79.2 ± 0.4% versus 73.6 ± 0.3% for sparse GCNNGP, below GCN’s 82.8 ± 0.5%.

Key insights

• Flexible representation learning, controlled by the `ν` parameter, is critical for achieving high performance on heterophilous graph datasets where direct graph structure is less indicative of node labels.
• For homophilous graphs, the inherent inductive bias of graph convolution (aggregating similar neighbors) is largely sufficient, with additional representation learning offering only marginal benefits.
• The GC-DKM framework successfully bridges the performance gap between fixed-kernel infinite-width models (NNGPs) and flexible finite-width neural networks (GCNs) by incorporating controlled representation learning within a principled kernel method.

Summary by alphaXiv.