Reproduce “MER-CLIP: AU-Guided Vision-Language Alignment for Micro-Expression Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shifeng Liu, Xinglong Mao, Sirui Zhao, Peiming Li, Tong Xu, Enhong Chen
MER-CLIP adapts the CLIP vision-language model for micro-expression recognition by using natural language descriptions of Action Units as semantic guidance. This approach yielded improved performance, including a UF1 score of 0.7832 on the CAS(ME)3 3-class classification task.
Key insights
- Converting Action Unit (AU) labels into natural language descriptions can effectively guide vision models to learn fine-grained spatiotemporal features of micro-expressions through cross-modal alignment.
- Specialized data augmentation strategies, such as LocalStaticFaceMix, can address data scarcity in micro-expression recognition by blending static facial features from onset frames to increase diversity while preserving subtle motion dynamics.
- A progressive training strategy that balances motion feature learning with emotion inference, coupled with a lightweight transformer head, improves the model's ability to map low-level facial movements to distinct emotion categories.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.