Reproduce “From Large to Super-Tiny: End-to-End Optimization for Cost-Efficient LLMs” on OpenResearch

About this paper

Jiliang Ni, Jiachen Pu, Zhongyi Yang, Kun Zhou, Hui Wang, Xiaoliang Xiao, Dakui Wang, Xin Li and 2 more

Alibaba Group researchers developed an end-to-end optimization pipeline for deploying cost-efficient large language models (LLMs) in real-world dialogue systems. This pipeline transforms large LLM prototypes into "super-tiny" models, achieving 180 times parameter compression and a 10x FLOPs reduction while maintaining near-state-of-the-art performance for domain-specific function-calling tasks.

Key insights

• Direct Reinforcement Learning is ineffective for enhancing complex reasoning in very small LLMs due to their limited capacity, but a sequential RL-KD-RL hybrid strategy successfully transfers advanced generalization and Chain-of-Thought capabilities.
• An LLM-as-a-Judge, optimized with a small expert-annotated dataset, can reliably evaluate prototype performance and filter high-quality training data with 93% consistency with human experts.
• "Super-tiny" LLMs (0.4-0.5 billion parameters) can achieve performance comparable to much larger models for specific domain tasks through systematic optimization, enabling massive cost and latency reductions.

Summary by alphaXiv.