Reproduce “Authorship attribution and aesthetic evaluation of AI poetry: a case study with Haiku” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Livia Oddi, Simone Scardapane, Toru Sugimoto, Donatella Genovese
A study of Japanese university students evaluating seasonal haiku finds that aesthetic plausibility does not reliably reveal authorship. Across six Japanese-capable language models, recognition was near chance for GPT-5, Gemini 2.5, and StableLM-7B, while participants’ impressions predicted perceived human authorship much better than the poems’ true source (AUC 0.793 versus 0.505).
Key insights
- Readers’ aesthetic impressions functioned as attribution heuristics: poems believed to be human-written received higher ratings, even though those ratings were nearly useless for identifying actual authorship.
- Detectability varied strongly by poem and model rather than following a universal AI signature. Recognition was approximately chance for GPT-5 (0.493), Gemini 2.5 (0.503), and StableLM-7B (0.524), but higher for LLaMA-2 (0.670).
- Formal compliance and cultural plausibility can make generated poetry difficult to distinguish from human work, so creative-AI evaluation should treat quality assessment and provenance recognition as separate tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.