Reproduce “SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check” on OpenResearch

About this paper

Xingyi Cheng, Weidi Xu, Kunlong Chen, Shaohua Jiang, Feng Wang, Taifeng Wang, Wei Chu, Yuan Qi

SpellGCN addresses Chinese spelling correction by using character-similarity graphs to shape classifiers alongside BERT’s contextual representations. Separate pronunciation and shape graphs influence character representations, and learned attention combines their outputs. On SIGHAN 2015, SpellGCN reaches 75.9% sentence-level correction F1, compared with 73.0% for BERT in the same training setting. This benchmark result does not isolate either graph’s individual contribution.

Key insights

• Explicitly modeling phonological and visual character similarities is crucial for effective CSC, as pre-trained language models alone cannot fully capture these nuances, leading to errors.
• Graph Convolutional Networks (GCNs) offer a robust and end-to-end trainable method to integrate structured symbolic knowledge, like character similarity graphs, into deep semantic representations.
• Dynamically combining information from multiple similarity graphs (e.g., pronunciation and shape) via an attention mechanism allows for adaptive character-specific correction based on context.

Summary by alphaXiv.