Reproduce “Task-conditioned adaptation of visual features in multi-task policy learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Pierre Marza, Laetitia Matignon, Olivier Simonin, Christian Wolf

A framework for multi-task robotic control introduces task-conditioned visual adapters that enable a single policy to adapt its perception to diverse tasks. This approach achieves an average 52.3% test success across benchmarks for known tasks, outperforming non-adapted single-task policies, and generalizes to 15 new tasks from only 5 demonstrations each by optimizing a task embedding.

Key insights

  • Modulating pre-trained visual features with task-conditioned adapters allows a single multi-task policy to effectively handle diverse tasks, often outperforming unadapted single-task policies.
  • Lightweight adapter modules provide an efficient mechanism for adapting powerful pre-trained vision models to specific robotic tasks, avoiding the computational cost and instability of full fine-tuning.
  • Optimizing a task embedding from a few demonstrations enables few-shot generalization to entirely new, unseen tasks, indicating the learned embedding space captures meaningful task regularities.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.