Reproduce “Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, Cho-Jui Hsieh
Cluster-GCN introduces an efficient algorithm for training deep Graph Convolutional Networks on large-scale graphs by employing graph clustering for mini-batch generation. The method enables training on datasets with millions of nodes and edges using significantly less memory, allows for the successful optimization of GCNs up to eight layers deep, and achieves new state-of-the-art accuracy on benchmarks like PPI (99.36% F1-score) and Reddit (96.60% F1-score).
Key insights
- Leveraging graph clustering to construct mini-batches dramatically improves "embedding utilization" and reduces the computational burden of neighborhood expansion, leading to linear complexity with network depth.
- Deeper GCNs are not inherently less effective; their previous performance degradation was largely due to optimization difficulties, which can be mitigated through techniques like diagonal enhancement.
- Stochastic mini-batching with multiple graph partitions can effectively balance computational efficiency with the need to capture inter-cluster dependencies and reduce gradient variance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.