Reproduce “EventHub: Data Factory for Generalizable Event-Based Stereo Networks without Active Sensors” on OpenResearch

About this paper

Luca Bartolomei, Fabio Tosi, Matteo Poggi, Stefano Mattoccia, Guillermo Gallego

EventHub, a framework for training deep event-based stereo networks without active sensors, leverages standard color images to generate proxy labels and events, demonstrating improved out-of-domain generalization. The framework also facilitates bi-directional knowledge transfer, enhancing RGB stereo models in low-light conditions and achieving near LiDAR-supervised performance for event stereo.

Key insights

• Leveraging abundant RGB data and state-of-the-art RGB depth estimation techniques can effectively overcome the scarcity of event-based training data, eliminating the reliance on active sensors.
• High-quality proxy labels and synthetic data, generated through NVS and cross-modal distillation, enable superior out-of-domain generalization for event stereo models compared to direct LiDAR supervision.
• Bi-directional knowledge transfer is feasible, allowing event-based models to be trained using RGB knowledge, and these event models can in turn enhance RGB stereo performance in challenging conditions like low-light.

Summary by alphaXiv.