Reproduce “Automatic Song Translation for Tonal Languages” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fenfei Guo, Chen Zhang, Zhirui Zhang, Qixin He, Kejun Zhang, Jun Xie, Jordan Boyd-Graber
A new unsupervised system, GagaST, translates songs for tonal languages like Mandarin by integrating pre-trained translation models with music-guided decoding constraints. The system improves the intelligibility and singability of translated songs by aligning linguistic tones with musical melodies, as demonstrated through human evaluation, while acknowledging a trade-off in literal meaning.
Key insights
- Successful automatic song translation for tonal languages necessitates explicitly aligning linguistic tones with musical pitches to ensure intelligibility.
- Combining unsupervised pre-training with a system of music- and language-specific constraints during decoding can effectively address the scarcity of parallel song translation data.
- Achieving singability and intelligibility in song translation often involves a trade-off with literal semantic fidelity, which is also observed in human-translated songs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.