Reproduce “Learning Graph Quantized Tokenizers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Limei Wang, Kaveh Hassani, Si Zhang, Dongqi Fu, Baichuan Yuan, Weilin Cong, Zhigang Hua, Hao Wu and 2 more
Meta AI introduces Graph Quantized Tokenizers (GQT), a framework that converts irregular graph data into discrete, compact tokens, allowing standard Transformer encoders to process graphs efficiently. GQT achieves state-of-the-art performance on 20 out of 22 benchmarks across various graph tasks while reducing memory usage by up to 30-fold.
Key insights
- Decoupling local interaction capture (via GNN-based tokenizer) from global dependency modeling (via Transformer) synergistically enhances graph representation learning.
- Residual Vector Quantization (RVQ) is effective for compressing graph information into a memory-efficient discrete token space, enabling scalability to large-scale graphs.
- Multi-task self-supervised learning for tokenization yields more robust and semantically rich representations, contributing to improved generalization and resilience to adversarial attacks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.