Reproduce “X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Di Cao, Dongjie Fu, Hai Yu, Siqi Zheng, Xu Tan, Tao Jin

X-OPD (Cross-Modal On-Policy Distillation) introduces a framework to align the reasoning and instruction-following capabilities of End-to-End Speech LLMs with text-based LLMs using on-policy sampling and dual-advantage functions. This approach reduced the average performance degradation for speech inputs to 3.43% and for text inputs to 0.97%, while also preventing catastrophic forgetting.

Key insights

  • Traditional fine-tuning and offline distillation methods were found to paradoxically worsen performance degradation in Speech LLMs, suggesting conflicts with internal priors and the limitations of static learning.
  • Effective cross-modal alignment requires a balanced approach, where both in-modal (text-to-text) and cross-modal (text-to-speech) advantage functions complement each other to preserve general capabilities.
  • Optimal teacher capacity for distillation may not always be the largest model; a teacher with a matched capacity can provide more effective and accessible learning trajectories for the student model.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.