Reproduce “TS-OPD: Reconciling ASR and QA in Speech Language Models via Task-Specific On-Policy Distillation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yujie Guo, Hongjie Chen, Jian Kang, Jie Li, Yongxiang Li, Yong Qin

TS-OPD adapts speech language models for transcription and question answering. The student learns from separate ASR and QA teachers, each supervising only its matching task, starting from a QA-capable model. It reports basic-ASR WER of 10.03% versus 14.19% for the QA model, and TELEVAL accuracy of 45.07% versus 42.57%; these findings are limited to the tested model and tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.