Reproduce “Structured Preference Optimization for Vision-Language Long-Horizon Task Planning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiwen Liang, Min Lin, Weiqi Ruan, Rongtao Xu, Yuecheng Liu, Jiaqi Chen, Bingqian Lin, Yuzheng Zhuang and 1 more

SPO trains a vision-language model for long-horizon household planning by ranking responses for textual coherence, image awareness, and final-action differences, with a curriculum from shorter to longer tasks. GCR measures achieved goal conditions, while SR requires all key conditions. On simulated ExtendaBench tests, SPO reaches 47.71% GCR and 16.83% SR in VirtualHome, outperforming Iterative RPO. These are simulator outcomes, not direct measures of reasoning quality or physical-robot performance.

Key insights

  • Decomposing reasoning quality into explicit, multimodal criteria (textual coherence and image awareness) for preference optimization provides more granular and effective feedback, enhancing embodied task planning beyond single-scalar reward approaches.
  • A targeted preference pair selection strategy, which considers not only overall preference scores but also differences in action outputs and reasoning quality, significantly contributes to more robust and accurate decision-making during optimization.
  • Progressive learning that incrementally exposes the model to increasing task complexity is crucial for developing robust and generalizable reasoning strategies, enabling consistent performance across diverse long-horizon scenarios.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.