Reproduce “NAC-TCN: Temporal Convolutional Networks with Causal Dilated Neighborhood Attention for Emotion Understanding” on OpenResearch

About this paper

Alexander Mehta, William Yang

Developed by researchers at the University of Pennsylvania, the NAC-TCN architecture enhances emotion understanding from video by combining Temporal Convolutional Networks with Causal Dilated Neighborhood Attention. It consistently achieves strong performance on emotion recognition datasets, for instance, matching Transformer accuracy on AffWild2 with 25x fewer parameters and 10x fewer MACs, demonstrating improved efficiency and contextual learning.

Key insights

• Explicitly enforcing causality within the temporal model improves performance, particularly for tasks with end-of-video labels, by preventing future information leakage.
• A hybrid architecture effectively combining efficient convolutional layers with localized attention mechanisms can achieve a balance between contextual understanding and computational cost.
• Residual connections are essential for ensuring training stability and maintaining performance in deep temporal convolutional networks with attention.

Summary by alphaXiv.