Reproduce “ConvFiT: Conversational Fine-Tuning of Pretrained Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ivan Vulić, Pei-Hao Su, Sam Coope, Daniela Gerz, Paweł Budzianowski, Iñigo Casanueva, Nikola Mrkšić, Tsung-Hsien Wen

ConvFiT adapts a pretrained language model for intent detection through conversational response-ranking fine-tuning followed by task-specific contrastive tuning. It represents utterances as vectors and predicts an intent by retrieving the nearest labeled example. In the paper’s 10-shot BANKING77 evaluation, the two-stage RoBERTa system with OCL reached 87.38% accuracy, above tested variants that omitted either stage, a result limited to this setup.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.