Reproduce “Pragmatic inference of scalar implicature by LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ye-eun Cho, Seong mook Kim
This research from Sungkyunkwan University investigates how Large Language Models like BERT and GPT-2 perform pragmatic inference, specifically scalar implicature. It demonstrates that while both models show a default pragmatic interpretation of "some" as "not all," GPT-2 exhibits human-like sensitivity to contextual cues, incurring higher processing difficulty when pragmatic inference is explicitly required by context, a pattern not observed in BERT.
Key insights
- Both BERT and GPT-2 display a default preference for pragmatic interpretation (e.g., "some" implies "not all") over semantic interpretation when no explicit context is provided.
- BERT's pragmatic processing aligns with the Default model, showing a consistent default implicature that is not significantly modulated by explicit contextual cues (QUDs).
- GPT-2's behavior aligns with the Context-driven model, demonstrating increased processing difficulty (higher surprisal) when the context explicitly demands the derivation of scalar implicature, mirroring human cognitive effort.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.