Reproduce “ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mouxiang Chen, Zemin Liu, Chenghao Liu, Jundong Li, Qiheng Mao, Jianling Sun
The ULTRA-DP framework unifies graph pre-training by introducing task and position-aware "dual prompts" through virtual "prompt nodes" into the input graph, effectively mitigating the semantic gap and task interference while incorporating position-specific knowledge without architectural changes. This approach yielded average relative performance gains of up to 4.4% over leading baselines across multiple few-shot learning settings.
Key insights
- Injecting explicit task and position information into the GNN's input space via virtual "prompt nodes" is a flexible and effective strategy to enhance graph pre-training without modifying the core architecture.
- Task embeddings enable GNNs to differentiate between various hybrid pre-training objectives, effectively mitigating task interference and fostering the acquisition of distinct, multi-grained knowledge.
- A prompt-based transferability test can successfully identify the most relevant pre-trained knowledge for downstream tasks, significantly improving knowledge transfer and reducing the semantic gap.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.