Reproduce “GLIP: Graph and LLM Joint Pretraining for Graph-Level Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoxin Sun, Yiqing Lin, Yajun Huang, Chenhui Dong, Mingjun Li, Zhongzhi Zhang
GLIP pretrains graph representations for graph-level tasks by turning whole-graph and selected subgraph information into tokens for a frozen LLM. It asks the LLM whether augmented views share an origin, while contrastive learning aligns views. On E-Com, GLIP-Mistral reaches 86.38% accuracy versus 81.23% for TEA-GLM-Mistral under graph-level adaptation; results cover tested settings, not every domain or task.
Key insights
- Large Language Models can serve as effective semantic decision interfaces to guide the self-supervised pretraining of graph encoders, aligning graph representations with LLM embedding spaces.
- Comprehensive graph understanding for LLMs requires presenting both global structural context and contextually enriched local substructures, which can be achieved through multi-token representation and diffusion.
- Joint pretraining significantly enhances the ability of smaller, frozen LLMs to perform complex graph reasoning and classification tasks with limited labeled data, reducing reliance on expensive fine-tuning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.