Reproduce “Efficient Event-Based Semantic Segmentation via Exploiting Frame-Event Fusion: A Hybrid Neural Network Approach” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hebei Li, Yansong Peng, Jiahui Yuan, Peixi Wu, Jin Wang, Yueyi Zhang, Xiaoyan Sun
A hybrid neural network framework effectively integrates frame-based and event-based data for semantic segmentation, achieving state-of-the-art accuracy and substantially reducing energy consumption. It leverages specialized interaction modules to facilitate dynamic feature exchange between parallel ANN and SNN branches.
Key insights
- Dynamic integration of SNN-derived temporal information into ANN features via the ATW Injector significantly enhances spatial representations.
- Incorporating rich color information from ANN features into SNN's temporal features, guided by event sparsity through the EDS Injector, improves SNN's feature extraction.
- Carefully designed channel selection and fusion mechanisms in the CSF module allow for effective combination of the most informative features from both branches, leading to refined semantic representation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.