Reproduce “Text Detoxification using Large Pre-trained Neural Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
David Dale, Anton Voronov, Daryna Dementieva, Varvara Logacheva, Olga Kozlova, Nikita Semenov, Alexander Panchenko
Researchers from Skoltech and MTS developed ParaGeDi and CondBERT, two unsupervised methods for text detoxification that leverage large pre-trained neural models. These methods achieved new state-of-the-art results in converting toxic text to non-toxic forms while prioritizing content preservation, confirmed by both automatic metrics and human evaluation.
Key insights
- Combining pre-trained paraphrasers with style-conditional guidance effectively balances content preservation and style transfer for detoxification.
- Pointwise editing models can achieve high performance by intelligently identifying and replacing toxic tokens with multi-word options, along with careful content preservation heuristics.
- Automatic content preservation metrics (e.g., embedding similarity) correlate weakly with human judgments, highlighting the limitations of current automatic evaluation for semantic fidelity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.