Reproduce “Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiaoze Liu, Ruowang Zhang, Weichen Yu, Siheng Xiong, Liu He, Feijie Wu, Hoin Jung, Matt Fredrikson and 2 more

Vision Wormhole (VW) routes latent messages between frozen vision-language models through pretrained visual-input pathways, avoiding prose handoffs. Each model uses a trained codec, while fitted affine maps align messages through shared anchors. Across heterogeneous teams, VW improves accuracy by an average of 6.0 percentage points and is 1.69× faster in evaluation runtime than text communication, though not every configuration improves.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.