Reproduce “Bidirectional Cross-Modal Prompting for Event-Frame Asymmetric Stereo” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ninghui Xu, Fabio Tosi, Lihui Wang, Jiawei Han, Luca Bartolomei, Zhiting Yao, Matteo Poggi, Stefano Mattoccia

The research introduces Bi-CMPStereo, a bidirectional framework for event-frame asymmetric stereo designed to integrate event camera dynamics and frame camera context for dense disparity estimation. It achieves superior accuracy and robust generalization in dynamic scenes and challenging illumination, surpassing both existing asymmetric and symmetric stereo methods on multiple benchmarks.

Key insights

  • Exploiting both modalities as both source and target in a bidirectional framework allows for high-fidelity cross-modal alignment without sacrificing the unique semantic and structural cues from each sensor.
  • Explicitly adapting source features to a target domain while enforcing reconstruction consistency in a canonical space effectively bridges the modality gap and prevents latent space collapse.
  • Regularizing context feature learning with hierarchical visual transformations enhances model robustness against shortcut learning, leading to superior generalization across diverse datasets and unseen scenarios.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.