Reproduce “ConvMixFormer- A Resource-efficient Convolution Mixer for Transformer-based Dynamic Hand Gesture Recognition” on OpenResearch

About this paper

Mallika Garg, Debashis Ghosh, Pyari Mohan Pradhan

IIT Roorkee researchers developed ConvMixFormer, an efficient deep learning architecture for dynamic hand gesture recognition, which replaces self-attention with a standard convolutional token mixer and integrates a Gated Depthwise Feed Forward Network. This model achieves competitive or superior performance on NVGesture and Briareo datasets while significantly reducing parameters and computational costs.

Key insights

• Standard convolutional layers can effectively serve as token mixers in transformer architectures, offering efficiency and superior ability to capture inter-channel dependencies and local spatial features critical for hand gesture recognition.
• The proposed Gated Depthwise Feed Forward Network (GDFN) enhances the model's capacity to selectively process information, yielding improved accuracy and parameter efficiency compared to standard FFNs.
• Significant reductions in model parameters and computational operations (MACs) are achievable in transformer-based models without compromising, and in some cases even improving, performance for complex vision tasks.

Summary by alphaXiv.