Reproduce “C-ICL: Contrastive In-context Learning for Information Extraction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ying Mo, Jiahao Liu, Jian Yang, Qifan Wang, Shun Zhang, Jingang Wang, Zhoujun Li
Researchers from Beihang University, Meituan, and Meta AI developed C-ICL, a contrastive in-context learning method for Information Extraction (IE) tasks like Named Entity Recognition (NER) and Relation Extraction (RE). This approach enhances few-shot Large Language Model (LLM) performance by incorporating both positive and carefully selected "hard negative" examples into prompts, demonstrating F1 score improvements up to +26.22% on relation extraction datasets like NYT.
Key insights
- Incorporating "wrong/negative" examples alongside "right/positive" ones in ICL demonstrations provides a more comprehensive learning signal for LLMs, moving beyond conventional positive-only conditioning.
- Carefully selected "hard negative samples" (predictions close to correct but wrong) combined with their correct labels enable LLMs to develop error awareness and learn to avoid common pitfalls.
- Effective retrieval strategies are crucial, with semantic similarity for positive samples and a combination of self-consistency and F1-score threshold for hard negative samples maximizing the utility of contrastive demonstrations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.