Reproduce “ConvFiT: Conversational Fine-Tuning of Pretrained Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ivan Vulić, Pei-Hao Su, Sam Coope, Daniela Gerz, Paweł Budzianowski, Iñigo Casanueva, Nikola Mrkšić, Tsung-Hsien Wen
ConvFiT adapts a pretrained language model for intent detection through conversational response-ranking fine-tuning followed by task-specific contrastive tuning. It represents utterances as vectors and predicts an intent by retrieving the nearest labeled example. In the paper’s 10-shot BANKING77 evaluation, the two-stage RoBERTa system with OCL reached 87.38% accuracy, above tested variants that omitted either stage, a result limited to this setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.