Reproduce “WaveDetect: Robust Framework for Machine-Generated Text Detection via Wavelet Transform” on OpenResearch

About this paper

Zhichen Liu, Kaitong Qin, Linhan He, Yang Xu

A new framework named WAVEDETECT identifies machine-generated text by analyzing token probability signals with a Continuous Wavelet Transform, achieving an average AUROC of 0.9785 and a True Positive Rate of 54.70% at 0.1% False Positive Rate on diverse benchmarks. This spectral approach demonstrates enhanced robustness against adversarial attacks and improved generalization across evolving language models and specialized domains.

Key insights

• Human and machine-generated texts possess distinct 'spectral fingerprints' in their underlying token probability distributions: human text exhibits stochastic, high-entropy fluctuations, while machine text shows smoother, more structured, and periodic patterns.
• The Continuous Wavelet Transform (CWT)'s multi-resolution analysis effectively captures fine-grained local transients and global trends in non-stationary probability signals, providing more robust features than global spectral methods like the Fourier Transform.
• The spectral features captured by WAVEDETECT are inherently invariant to surface-level semantic variations and domain shifts, leading to superior robustness against adversarial attacks, evolving LLMs, and diverse topics.

Summary by alphaXiv.