Reproduce “DenseTwins: Dense Modality Decoupling for Multimodal Self-Supervision” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Carlo Sgaravatti, Daniele Kopyshevskiy, Davide Secco, Giacomo Boracchi
DenseTwins learns dense spatial features from paired sensors by matching local correspondences and separating shared information from modality-specific cues. It uses redundancy reduction within views and aligns shared features across modalities. On SUN-RGBD frozen NYU-13 probing it reaches 40.6 mIoU versus DeCUR’s 35.8; on EventScape it scores 63.3 mIoU, below CLIP’s 63.5, and requires synchronized paired observations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.