Reproduce “Language is All a Graph Needs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, Yongfeng Zhang

Large Language Models (LLMs) can serve as foundational models for graph machine learning by representing graph structures purely through natural language instructions. The proposed InstructGLM framework consistently outperforms state-of-the-art Graph Neural Networks and graph Transformers on node classification benchmarks, especially under low-label conditions.

Key insights

  • Natural language is sufficiently expressive to convey complex graph topological information directly to LLMs, enabling them to effectively reason about graph structures.
  • Generative LLMs, through instruction tuning on natural language graph descriptions, can implicitly capture and leverage GNN-like inductive biases such as multi-hop aggregation and path information.
  • The 'everything is tokenized' vision extends effectively to graph data, positioning LLMs as versatile foundation models capable of unifying graph processing within a language-centric AI paradigm.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.