Reproduce “Bayesian Optimization with LLM-Based Acquisition Functions for Natural Language Preference Elicitation” on OpenResearch

About this paper

David Eric Austin, Anton Korikov, Armin Toroghi, Scott Sanner

Researchers from the University of Toronto and University of Waterloo developed PEBOL, a framework that integrates Bayesian Optimization with Large Language Models for efficient natural language preference elicitation in conversational recommender systems. PEBOL significantly outperforms monolithic LLM baselines in cold-start settings, achieving up to 0.27 Mean Reciprocal Rank at 10 (MRR@10) on Yelp, compared to GPT-MonoLLM's 0.12, by strategically guiding LLM query generation with decision-theoretic policies.

Key insights

• Combining formal Bayesian Optimization policies with LLMs' natural language capabilities enables strategic, multi-turn preference elicitation that overcomes the limitations of both monolithic LLMs and traditional BO.
• Probabilistic Natural Language Inference (NLI) scores can effectively translate nuanced natural language feedback into quantitative signals for robust Bayesian belief updates.
• Strategically selecting a small, relevant context for LLMs to generate queries, rather than providing the entire item set, improves efficiency, manages context windows, and allows for principled exploration-exploitation.

Summary by alphaXiv.