Reproduce “Comparing Graph Transformers via Positional Encodings” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mitchell Black, Zhengchao Wan, Gal Mishne, Amir Nayyeri, Yusu Wang

This paper rigorously compares Absolute and Relative Positional Encodings for Graph Transformers using novel Weisfeiler-Lehman test variants. It theoretically demonstrates that these encoding types offer equivalent distinguishing power for graphs without node features, while indicating that RPEs may provide an advantage for graphs with node features if an RPE-to-APE conversion is used.

Key insights

  • Absolute Positional Encodings (APEs) and Relative Positional Encodings (RPEs) are theoretically equivalent in distinguishing non-isomorphic graphs when no node features are present.
  • Converting RPEs to APEs via 2-equivariant graph networks can strictly decrease distinguishing power for graphs with initial node features.
  • Combinatorially-aware RPEs (e.g., Shortest-Path Distance) can ensure Graph Transformers achieve at least the same expressive power as Message Passing Neural Networks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.