Reproduce “Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zijun Wu, Yongkang Wu, Lili Mou
The authors introduce PTfer, which transfers soft prompts by representing each learned prompt vector through its cosine similarities to shared vocabulary anchors, then searching target-model vectors with matching relationships. On LAMA-TREx, a factual benchmark testing object prediction from subject–relation–object facts, BERT-base to BERT-large transfer reaches 31.40% accuracy, above a neural-projector baseline but below manual prompting and direct target tuning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.