Reproduce “EVIL-Detect for NLPCC 2026 Shared Task 6: LLM-Generated Text Detection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hongrui Bao, Hangyu Rong, Zhuoshang Wang, Yubing Ren, Yanan Cao
EVIL-Detect is a multi-signal ensemble framework developed for robust three-class Chinese text detection, distinguishing between human-written, LLM-generated, and LLM-refined content. The system achieved first place in NLPCC 2026 Shared Task 6 with a macro-F1 score of 0.8888, demonstrating consistent performance on out-of-distribution evaluation data.
Key insights
- No single detection signal is sufficient for robust three-class text detection in challenging out-of-distribution (OOD) Chinese language settings.
- LLM-refined text (HLT) is the most challenging class to detect due to its intermediate nature between human-written and LLM-generated text, requiring specialized multi-signal strategies.
- Integrating diverse, complementary signals via a conflict-aware fusion strategy significantly enhances overall detection accuracy and robustness, especially for resolving ambiguities around the HLT category.
- Framing AI-edited text detection as a continuous estimation of "edit-extent" provides a valuable approach for handling varying degrees of AI intervention.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.