Reproduce “Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinlin Peng, Ying Zhou, Ben He, Le Sun, Yingfei Sun
This paper evaluates the vulnerability of AI-generated content detection systems against adversarial attacks tailored for student essays, demonstrating that simple word and sentence substitutions can significantly reduce detection accuracy while maintaining essay quality. It also introduces AIG-ASAP, a new dataset for benchmarking AIGC detectors specifically for educational settings.
Key insights
- Current AI-generated content detectors, while effective on unperturbed AI essays, are highly vulnerable to adversarial word and sentence substitutions.
- Detection evasion is possible while maintaining essay quality, as evidenced by human evaluators often preferring perturbed AI-generated essays over human-written ones.
- The effectiveness of perturbation methods and detector resilience varies across different essay types, with argumentative essays being more susceptible to evasion than source-dependent ones.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.