Reproduce “Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Models” on OpenResearch

About this paper

Shuang Li, Jiangjie Chen, Siyu Yuan, Xinyi Wu, Hao Yang, Shimin Tao, Yanghua Xiao

Li et al. developed IDIOMKB, a large-scale multilingual idiom knowledge base distilled from LLMs, and introduced a KB-CoT prompting method that significantly improves idiomatic translation quality for smaller language models. The research also established a GPT-4-powered evaluation metric that strongly correlates with human judgments, outperforming traditional MT metrics in assessing idiomatic accuracy.

Key insights

• Leveraging LLMs for "symbolic knowledge distillation" is an effective way to create comprehensive, multilingual idiom knowledge bases, demonstrating high quality in generated figurative meanings.
• Integrating external, pre-computed idiom meanings via KB-CoT prompting consistently and substantially improves idiomatic translation quality across various language models, including smaller ones.
• Large Language Models like GPT-4 can be effectively utilized as robust, human-aligned evaluation metrics for nuanced tasks such as idiomatic translation, where traditional metrics are inadequate.

Summary by alphaXiv.