Reproduce “Watermarking Text Generated by Black-Box Language Models” on OpenResearch

About this paper

Xi Yang, Kejiang Chen, Weiming Zhang, Chang Liu, Yuang Qi, Jie Zhang, Han Fang, Nenghai Yu

This research from the University of Science and Technology of China, Nanyang Technological University, and National University of Singapore introduces a framework for watermarking text generated by black-box large language models. The method subtly modifies existing text through context-aware synonym substitution to embed a detectable statistical signal, achieving high detection accuracy while preserving semantic fidelity across English and Chinese texts.

Key insights

• Watermarking can be achieved in black-box LLM scenarios by subtly modifying generated text via context-aware synonym substitution, rather than internal model manipulation.
• A binary encoding function tied to word and preceding word hashes allows for statistical detection of watermarks without requiring access to LLM internals or output probabilities.
• Maintaining high semantic fidelity during watermark injection is crucial, and the paper demonstrates that it is challenging for attackers to remove the watermark without significantly degrading the text's quality.

Summary by alphaXiv.