Reproduce “Event Temporal Relation Extraction based on Retrieval-Augmented on LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaobin Zhang, Liangjun Zang, Qianwen Liu, Shuchong Wei, Songlin Hu
Researchers from the Chinese Academy of Sciences developed a Retrieval-Augmented Generation (RAG) framework to automate the design of prompt templates and verbalizers for Event Temporal Relation (TempRel) extraction. Their PETR model outperformed existing baselines on TB-Dense, TDD-Man, and TDD-Auto datasets, improving the F1 score for the "SIMULTANEOUS" relation from 0% to 36.4%.
Key insights
- LLMs can act as effective "prompt engineers" by generating and optimizing prompt templates and verbalizers, reducing reliance on manual design and enhancing performance for complex tasks like TempRel extraction.
- The specific design of prompt components, including template style (e.g., post-decorated) and modifier words (e.g., "trigger"), as well as the use of soft verbalizers, critically impacts TempRel extraction accuracy.
- Integrating RAG effectively infuses richer temporal knowledge into prompt design, helping to resolve the inherent ambiguity in temporal relations and leading to more accurate and nuanced relation classification.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.