Reproduce “A Simple yet Efficient Ensemble Approach for AI-generated Text Detection” on OpenResearch

About this paper

Harika Abburi, Kalyani Roy, Michael Suesserman, Nirmala Pudota, Balaji Veeramani, Edward Bowen, Sanmitra Bhattacharya

Researchers at Deloitte developed a simplified ensemble method for AI-generated text detection, utilizing only two pre-trained language models to achieve performance comparable to or exceeding state-of-the-art baselines. The approach also demonstrates that open-source LLM data like LLaMA2 can effectively replace proprietary GPT data for training robust detectors and shows a comparative advantage in classifying non-native English writing over several commercial tools, despite some remaining bias.

Key insights

• A condensed ensemble architecture comprising only two constituent models can deliver state-of-the-art AI-generated text detection performance with reduced computational complexity and increased efficiency.
• Open-source LLM data, specifically LLaMA2-generated text, can effectively serve as a viable and potentially superior alternative to proprietary GPT data for training generative text detectors.
• While the non-perplexity-based ensemble still exhibits some misclassification for non-native English writing, it shows a comparative advantage over several commercial AI-generated text detection tools in this regard.

Summary by alphaXiv.