Reproduce “Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Cong Zeng, Shengkun Tang, Yuanzhou Chen, Zhiqiang Shen, Wenchao Yu, Xujiang Zhao, Haifeng Chen, Wei Cheng and 1 more
Researchers from MBZUAI, UCLA, and NEC Lab introduced a new paradigm for detecting LLM-generated text by reframing it as an out-of-distribution (OOD) detection problem, treating machine text as in-distribution and human text as OOD. This approach yielded superior generalizability and robustness across multilingual, adversarial, and unseen domain/model scenarios, achieving up to 98.3% AUROC on DeepFake and 96.4% AUROC on M4-multilingual datasets, while also demonstrating improved efficiency.
Key insights
- Human language's inherent diversity makes it unsuitable to be modeled as a single, coherent distribution for binary classification.
- Machine-generated text, due to its more consistent statistical patterns, is better modeled as a compact "in-distribution" manifold.
- Treating human text as OOD naturally accounts for its vast, unpredictable variability, leading to detectors that generalize better to unseen human writing styles and novel LLM outputs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.