Reproduce “Why is AI-generated language so hard to understand?” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Thomas Hikaru Clark, Matias B. Martin, Kyle Mahowald
The authors compare model text in chat, coding, and essays with round-trip translation paraphrases, measuring features associated with human processing difficulty. Original wording often contains more such features, but the study does not test reader comprehension. A length-penalty task drove shorter choices; two constructions exceeded 98% without KL regularization. This does not establish the cause of frontier-model patterns.
Key insights
- Many frontier and near-frontier models used more low-frequency sequences and several constructions associated with processing costs than their round-trip paraphrases; patterns varied by genre and model family.
- Models often favored omitting optional words, while effects for long dependencies and stacked nouns were less consistent, particularly in essays.
- A length penalty can push models toward shorter grammatical alternatives even when task correctness is maintained; transfer was clear for related optional-*that* cases but limited for more distant constructions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.