Reproduce “Model Generalization on Text Attribute Graphs: Principles with Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haoyu Wang, Shikun Liu, Rongzhe Wei, Pan Li

The paper introduces LLM-BP, a training-free framework that enables robust zero-shot inference over Text-Attributed Graphs by leveraging large language models to generate task-adaptive embeddings and dynamically guide a Belief Propagation algorithm for graph information aggregation. This approach achieves top performance across diverse homophilic and heterophilic graph datasets without requiring any target graph training.

Key insights

  • Many existing LLM-based methods that use graph adapters to align graph embeddings with LLM token spaces often underperform simpler LM encoders, indicating insufficient training data for effective alignment.
  • Large Language Models can function as intelligent analytical agents to deduce and parameterize graph-theoretic properties like homophily, enabling adaptive algorithmic decisions for structural inference.
  • Task-adaptive prompting significantly enhances the discriminative power of LLM-induced node embeddings, leading to more distinct class clustering and improved classification performance without further training.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.