Reproduce “Post-Hoc Understanding of Metaphor Processing in Decoder-Only Language Models via Conditional Scale Entropy” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lawhori Chakrabarti, Jennifer Johnson-Leung, Bert Baumgaertner, Aleksandar Vakanski, Min Xian, Boyu Zhang
Researchers from the University of Idaho developed Conditional Scale Entropy (CSE), a wavelet-derived measure, to analyze metaphor processing in decoder-only language models. Their analysis reveals that metaphorical language consistently induces higher multi-scale coordination (elevated CSE) in the early-to-mid layers of these models compared to literal language, providing a robust computational signature.
Key insights
- Metaphorical processing consistently engages broader multi-scale coordination (higher CSE) in transformer models compared to literal processing.
- This distinct multi-scale engagement is localized to specific 'active zones' within the early-to-mid layers across various decoder-only architectures, suggesting a common computational phase for figurative language understanding.
- Conditional Scale Entropy provides a robust, magnitude-invariant tool for mechanistic interpretability, characterizing the organizational structure of internal computation rather than just its intensity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.