Reproduce “RM-VLM: From Natural-Language Task Descriptions to Vision-Grounded Reward Machines” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Roko Parac, Christos Ziakas, Leo Ardon, Alessandra Russo

RM-VLM uses an LLM to generate reward machines from task instructions, and a vision-language model to ground their propositions in images. It tracks uncertain progress over machine states using estimated label error rates. On OGBench-Scene’s drawer-and-window task, offline IQL achieved 0.98 ± 0.02 versus 0.73 ± 0.15 for direct VLM reward detection. Policy experiments used expert specifications; generated machines were trace-equivalent in 114/120 runs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.