Reproduce “Intra-Modal Neighbors Never Lie: Rectifying Inter-Modal Noisy Correspondence via Graph-Based Intra-Modal Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yang Liu, Wentao Feng, Shu-Dong Huang, Yalan Ye, Jiancheng Lv
A framework called IN²R (Intra-modal Neighbor-aware Noise Rectification) addresses noisy image-text correspondence by synthesizing continuous semantic prototypes from intra-modal neighbors using graph-based reasoning. The method achieved state-of-the-art cross-modal retrieval performance, improving rSum by 21.3 points over baselines on Flickr30K with 80% noise.
Key insights
- Despite noisy explicit inter-modal pairings, the intrinsic geometric structure and semantic proximity within each modality remain stable and reliable, serving as a robust source for noise rectification.
- Synthesizing a continuous semantic prototype from the collective consensus of a local neighborhood provides more robust and precise supervision compared to selecting a single discrete substitute.
- Decoupling the source of rectification (memory bank) from the querying model prevents confirmation bias, ensuring unbiased rectification signals.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.