Reproduce “The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking” on OpenResearch

About this paper

Yinghui Li, Qingyu Zhou, Yangning Li, Zhongli Li, Ruiyang Liu, Rongyi Sun, Zizhen Wang, Chao Li and 2 more

A framework called Error-driven Contrastive Probability Optimization (ECOPO) refines pre-trained language models for Chinese Spell Checking by leveraging their past incorrect high-probability predictions as negative feedback. The method consistently improves correction F1 scores across SIGHAN benchmarks, achieving 78.4% on SIGHAN15 with a BERT encoder, and enhances other advanced models without requiring additional linguistic knowledge.

Key insights

• PLMs' bias towards common characters in CSC can be directly addressed by optimizing the prediction probabilities for specific confusing characters against high-probability incorrect alternatives.
• Leveraging the model's own incorrect (but highly confident) predictions as explicit negative feedback effectively guides its learning, embodying the "past mistake is future wisdom" principle.
• The proposed method is model-agnostic, allowing it to enhance various existing PLM-based CSC architectures without requiring complex architectural changes or external linguistic knowledge.

Summary by alphaXiv.