Reproduce “CULTURE-CMT: Benchmarking Contextualized Social Media Translation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Linjuan Wu, Dingming Li, Yiwen Qiu, Xinze Lyu, Ye Guo, Daoxin Zhang, Zhe Xu, Weiming Lu
The authors evaluate Chinese-to-English social-media comment translation with post and parent-comment context, separating slang, context-dependent cases, and examples combining both. The benchmark contains 572 manually verified examples and uses Invalid Rate, where lower is better. With context and thinking inference, the adapted 27B model records 17.7%, the lowest observed rate among evaluated configurations; this is benchmark-specific, not platform-wide.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.