Reproduce “GrokFormer: Graph Fourier Kolmogorov-Arnold Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guoguo Ai, Guansong Pang, Hezhe Qiao, Yuan Gao, Hui Yan
GrokFormer combines efficient global self-attention with a Graph Fourier Kolmogorov-Arnold filter that adapts both across Laplacian frequencies and spectral orders. It learns low-pass, high-pass, band-pass, and comb-shaped responses, achieving the best reported node-classification accuracy on 10 of 11 datasets and strong results on graph classification, while reducing reported memory and runtime relative to comparable spectral Transformers.
Key insights
- The learned spectral behavior varies with graph structure: filters are mainly low-pass on homophilic datasets, high-pass on Texas, and comb-like on datasets such as Squirrel and Penn94.
- Synthetic filter-fitting experiments show that the method can closely approximate irregular responses, including comb and band-rejection filters, with errors from 0.0004 to 0.0029 and approximately 0.9999 R².
- Spectral filtering and global attention are complementary: the full model generally outperforms either component alone, although stronger spectral expressiveness does not guarantee the best result on every large graph benchmark.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.