Reproduce “Multilingual Emotion Neurons in Large Audio-Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiutian Zhao, Philipp Koehn, Björn Schuller, Berrak Sisman
This research mechanistically explores how Large Audio-Language Models (LALMs) process emotion across different languages, introducing Consistency-Regularized Fusion (CR-Fusion) to pinpoint Multilingual Emotion Neurons (MLENs). These identified neurons demonstrated stable emotional selectivity and consistently aligned causal effects, yielding superior transferable affective control in zero-shot and low-resource settings compared to monolingual methods.
Key insights
- Monolingually identified emotion-sensitive neurons show low set intersection across languages, indicating that traditional methods struggle to isolate truly transferable affective units despite underlying shared selectivity patterns.
- The primary bottleneck for identifying transferable affective units is the linguistic diversity of the identification data rather than the sheer volume of within-language samples, emphasizing the need for cross-lingual evidence integration.
- LALMs possess an identifiable 'language-invariant emotion effect' which accounts for 30% to 59% of emotion-conditioned activation structure, and fusion strategies like CR-Fusion can effectively recover this invariant component.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.