Reproduce “PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lorenzo Molfetta, Giacomo Frisoni, Nicolò Monaldini, Gianluca Moro
The authors introduce PORTS, a tool retriever trained to align rankings with feedback from a frozen language model and favor correct tools over distractors. It combines REPLUG-style guidance with a preference signal. Across six datasets and three language models, RoBERTa’s average Recall@{1,2,3} rises from 50.8% with REPLUG to 57.5% with PORTS; these are retrieval results, and gains depend on documentation quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.