Reproduce “On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion” on OpenResearch

About this paper

Chenghao Fan, Zhenyi Lu, Wei Wei, Jie Tian, Xiaoye Qu, Dangyang Chen, Yu Cheng

Researchers at Huazhong University of Science and Technology developed Dynamic Logit Fusion, a method that efficiently specializes large language models by dynamically transferring knowledge from multiple fine-tuned small models without retraining the larger model. This approach closed up to 96.4% of the performance gap with full 13B LLM fine-tuning and demonstrated superior generalization on unseen tasks, outperforming static logit arithmetic baselines.

Key insights

• The optimal ratio for transferring specialized knowledge from small models to large models is dynamic, varying at each decoding step, and adaptively adjusting it improves performance significantly.
• Fusing knowledge from multiple specialized small language model experts enhances generalization and robustness, particularly for multi-task and unseen scenarios.
• The proposed KL divergence-based optimization framework effectively aligns the distribution shifts between base and fine-tuned small models with the dynamically fused large model, providing a theoretical foundation for adaptive knowledge transfer.

Summary by alphaXiv.