Reproduce “Shared Geometry As A Rosetta Stone: Cross-Modal Alignment Without Paired Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dominik Schnaus, Thomas Dagès, Daniel Cremers, Xi Wang, Phillip Isola
The authors introduce an unpaired method aligning image and text embedding spaces by matching cluster-similarity patterns and refining a map. On MS COCO, it achieves average FOSCTTM of 0.154 across 21 model combinations, compared with 0.223 for mini-vec2vec. This indicates coarse correspondence, not exact sample-wise recovery; success depends on shared geometry, which cannot be assessed from wholly unpaired data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.