Reproduce “RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yijia Fan, Zonglin Di, Zimo Wen, Yifan Yang, Mingxi Cheng, Qi Dai, Bei Liu, Kai Qiu and 3 more

Resource2Skill presents a framework to automatically distill executable skills for software agents from diverse human-created multimodal resources, particularly tutorial videos. The system improved agent performance by an average of +11.9 percentage points across seven software-authoring domains, and human evaluators favored its outputs in 85.5% of non-tied ratings.

Key insights

  • Distilling procedural knowledge from multimodal human-created resources significantly enhances agent capabilities for complex software-authoring tasks.
  • A hierarchical organization and multimodal skill representations (text, visual, code) provide substantial performance benefits over flat, text-only skill libraries.
  • Online skill acquisition effectively fills capability gaps, enabling agents to dynamically extract and apply new skills from resources for novel task requirements.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.