Reproduce “Composable Soft-Prompt Distillation of RL Experts: Toward Continual Post-Training” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhen Zhang, Changyi Yang, Xuehai He, Chengzhi Liu, Yutong Yin, Ziyue Liu, Zhuoran Yin, Yangmin Li and 20 more
CSPD addresses continual reinforcement-learning post-training by distilling each expert separately into a soft prompt while keeping the pre-training backbone frozen, then concatenating prompts at inference. This lets experts be added or removed without retraining shared weights. In a four-domain experiment, the combined prompts raised the average from 50.41 to 67.83, essentially matching MOPD’s 67.79; results are limited to this setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.