Reproduce “Language Models at the Syntax-Semantics Interface: A Case Study of the Long-Distance Binding of Chinese Reflexive ziji” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiulin Yang

Georgetown University research evaluated language models' ability to process the intricate syntax-semantics of Chinese reflexive ziji's long-distance binding, finding that no model consistently replicated human judgments. Models displayed linear biases and showed more sensitivity to noun semantics (animacy) than verb semantics (reflexiveness), with larger models not necessarily performing better.

Key insights

  • Language models do not consistently acquire human-like understanding of complex, abstract linguistic rules like ziji binding, despite their impressive performance on many NLP tasks.
  • Model performance is significantly influenced by linear biases (e.g., favoring local vs. long-distance binders), which vary across encoder-only and decoder-only architectures.
  • Models demonstrate an uneven acquisition of semantic knowledge, showing greater sensitivity to noun-related semantics (animacy) compared to verb-related semantics (reflexiveness).

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.