Reproduce “Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hy Dang, Tianyi Liu, Zhuofeng Wu, Jingfeng Yang, Haoming Jiang, Tao Yang, Pei Chen, Zhengyang Wang and 4 more

Researchers from the University of Notre Dame and Amazon developed a guided-structured template framework to enhance large language models' accuracy, robustness, and interpretability in function-calling tasks. This approach, applied through both structured prompting and supervised fine-tuning with a new ToolGT dataset, led to consistent performance improvements across diverse models and benchmarks.

Key insights

  • Explicit, structured reasoning templates consistently outperform free-form Chain-of-Thought (CoT) and direct function calling (No Thought) in function-calling tasks, demonstrating the need for guided scaffolding in structured applications.
  • Internalizing structured reasoning through supervised fine-tuning on a dedicated dataset like ToolGT consistently yields the highest performance, especially for smaller models struggling with structured prompting alone.
  • The 'Detail' template, providing more explicit and comprehensive guidance, achieved higher overall accuracy than simpler templates, indicating that task-specific complexity in templates can be beneficial.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.