Reproduce “Data-efficient Event Camera Pre-training via Disentangled Masked Modeling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhenpeng Huang, Chao Li, Hao Chen, Yongjian Deng, Yifeng Geng, Limin Wang
Researchers from Southeast University and Alibaba Group developed a data-efficient self-supervised learning method for event cameras, using a voxel-based representation and a disentangled masked modeling approach. This pre-training strategy achieves 88.0% accuracy on N-Caltech101 object recognition, outperforming prior methods like ECDP by 2.6 percentage points without requiring paired RGB images and showing faster convergence from limited data.
Key insights
- Directly processing event data in a voxel-based format, coupled with a multi-scale transformer encoder, effectively captures high-speed motion and spatio-temporal information.
- Disentangling the reconstruction task into local and global objectives allows the encoder to learn both fine-grained intra-cluster details and high-level inter-cluster semantics efficiently.
- Semantic-uniform masking, by ensuring balanced sampling from sparse and dense regions, significantly improves learning efficiency and model performance on non-uniform event data compared to random masking.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.