Reproduce “PEToolLLM: Towards Personalized Tool Learning in Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qiancheng Xu, Yongqi Li, Heming Xia, Fan Liu, Min Yang, Wenjie Li
Researchers developed PEToolLLaMA, a two-stage training framework, and PEToolBench, the first benchmark, to enable large language models to learn and apply tools based on individual user preferences inferred from interaction history. The framework demonstrated over 75% relative improvement in Tool Accuracy compared to existing models, achieving 77.83% overall accuracy on the new benchmark.
Key insights
- Current state-of-the-art LLMs, including GPT-4o, exhibit limited ability in personalized tool selection, particularly with complex or evolving user interaction histories.
- The primary challenge in personalized tool learning lies more in selecting the correct tool based on inferred preferences rather than accurately generating its parameters once selected.
- A two-stage training approach combining Supervised Fine-Tuning and Direct Preference Optimization is effective in teaching LLMs to both understand tool functionality and align with implicit user preferences.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.