Reproduce “Ontology-enhanced Prompt-tuning for Few-shot Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hongbin Ye, Ningyu Zhang, Shumin Deng, Xiang Chen, Hui Chen, Feiyu Xiong, Xi Chen, Huajun Chen
OntoPrompt is a framework that enhances few-shot learning by injecting external ontology knowledge into pre-trained language models via prompt-tuning. It addresses challenges of knowledge missing, noise, and heterogeneity through an ontology transformation module, a span-sensitive knowledge injection mechanism, and a collective training algorithm, demonstrating substantial performance improvements across relation extraction, event extraction, and knowledge graph completion tasks in low-resource settings.
Key insights
- Transforming structured ontology knowledge into textual prompts effectively bridges the gap between symbolic knowledge and pre-trained language models.
- A span-sensitive attention mechanism is crucial for filtering noisy or irrelevant external knowledge, ensuring that only informative knowledge related to key text spans is injected.
- Jointly optimizing both the pre-trained language model and the ontology-derived prompt tokens (via collective training) leads to a more cohesive and robust integration of knowledge.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.