Reproduce “Draft-KV: Learning Useful Latent Communication Between Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Linquan Wu, Shichang Meng, Tianxiang Jiang, Haoyu Yang, Peng Zhong, Fengming Zhu, Xi Peng, Linqi Song and 2 more

The authors evaluate whether latent messages between language models carry question-specific information and introduce Draft-KV, which sends a sharer’s draft-position key–value states through a trained bridge to a frozen receiver. They find earlier interfaces’ pairing gains no greater than 0.60 percentage points across five method–dataset pairs, while Draft-KV reaches 78.04% on MMLU-Redux versus 36.40% with reassigned messages; results are setting-specific.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.