Reproduce “Unsupervised Domain Clusters in Pretrained Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Roee Aharoni, Yoav Goldberg
The paper presents two ways to select training examples for a new task from a large corpus using a small set of target-domain sentences: cosine retrieval with pretrained language-model representations and a fine-tuned binary classifier. In German-to-English translation, Domain-Finetune selected 500,000 examples and averaged 41.26 SacreBLEU, versus 40.88 for Moore–Lewis, across five domains on this benchmark.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.