Reproduce “Efficiently Generating Expressive Quadruped Behaviors via Language-Guided Preference Learning” on OpenResearch

About this paper

Jaden Clark, Joey Hejna, Dorsa Sadigh

This research introduces Language-Guided Preference Learning (LGPL), a method for efficiently generating expressive quadruped behaviors by combining large language models (LLMs) with preference learning. The approach leverages LLMs to provide informed initial candidate behaviors, which are then rapidly refined through a small number of human preference queries, demonstrated to achieve higher accuracy and better human alignment with fewer interactions compared to existing methods.

Key insights

• Large language models (LLMs) can generate high-quality initial candidate robot behaviors, providing strong priors that significantly reduce the data required for subsequent preference learning.
• Augmenting human preference data by breaking trajectories into sub-segments drastically improves the sample efficiency of preference learning, especially in few-shot settings.
• A hybrid approach combining the high-level semantic understanding of LLMs with the high-resolution feedback of preference learning enables efficient and precise adaptation of complex robot behaviors.

Summary by alphaXiv.