Reproduce “STAR: Similarity-guided Teacher-Assisted Refinement for Super-Tiny Function Calling Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiliang Ni, Jiachen Pu, Zhongyi Yang, Jingfeng Luo, Conggang Hu

Researchers at Alibaba developed STAR, a framework for super-tiny function calling models that integrates Constrained Knowledge Distillation (CKD) and Similarity-guided Reinforcement Learning (Sim-RL). This approach enables models as small as 0.6B parameters to achieve 51.70 overall accuracy on BFCLv3 and 53.00 on ACEBench, demonstrating improved generalization and outperforming larger, 8B parameter models.

Key insights

  • Combining top-k truncation with mode-seeking Reverse KL (RKL) in KD leads to training collapse and diminished exploratory capacity, whereas Forward KL (FKL) is more stable but benefits from additional regularization.
  • Constrained Knowledge Distillation (CKD) effectively re-balances learning signals to prevent training instability and maintain high policy entropy, providing a stronger initialization for subsequent RL.
  • A fine-grained, continuous reward function, as implemented in Sim-RL, is more effective than binary rewards for function calling tasks, enabling partial credit and robust generalization by reducing sensitivity to syntactic variations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.