Reproduce “CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonese” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dazhong Chen, Yi-Cheng Lin, Yuchen Huang, Ziwei Gong, Di Jiang, Zeying Xie, Yi R., Fung
A framework called CantoASR enhances Automatic Speech Recognition for low-resource Cantonese by integrating acoustic-prosodic cues with phonological reasoning within a collaborative ASR-LALM system. It achieves an 11.19% Character Error Rate on spontaneous Cantonese datasets, reducing errors by about 3.3 absolute percentage points compared to leading LALM baselines.
Key insights
- Explicitly grounding LALM-based error correction in acoustic-prosodic measurements through instruction tuning is effective for tonal languages, even without manual tone labels.
- Complementary benefits are achieved by combining ASR finetuning for acoustic robustness with LALM instruction tuning for phonological reasoning.
- Strategic inference, including confidence-based filtering and multi-stage correction validated by a lightweight Cantonese LLM, reduces overcorrection and improves tonal accuracy.
- Constrained decoding strategies, such as N-best and lattice constraints, further enhance error correction performance by leveraging ASR-derived hypotheses.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.