Reproduce “Empirical Analysis of Dialogue Relation Extraction with Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guozheng Li, Zijie Xu, Ziyu Shang, Jiajun Liu, Ke Ji, Yikai Guo
An empirical analysis of Dialogue Relation Extraction (DRE) with large language models (LLMs) reveals that both proprietary and fine-tuned open-source LLMs achieve state-of-the-art performance. The study demonstrates that these models effectively capture long-range dependencies and maintain robustness in partial dialogue contexts, surpassing existing sequence-based and graph-based methods.
Key insights
- Indirect prompting strategies, which decompose DRE into a question-answering process, are more effective for proprietary LLMs than direct, vanilla prompting.
- Scaling open-source generative models within the Landre framework, particularly with LoRA, yields substantial performance gains for DRE tasks (over 20% absolute F1 from GPT-2 to LLaMA 7B).
- LLMs exhibit significantly enhanced robustness to varying dialogue lengths and partial contexts, demonstrating minimal performance drops between full and partial dialogue settings compared to prior methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.