Reproduce “TaskFrame: Connecting Task-Level Planning with Frame-Level Control for Game Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Guangzhi Wang, Junfu Pu, Yiran Wang, Xingyilang Yin, Honghao Chen, Cheng Cheng, Lingen Li, Yinan Zhou and 5 more

TaskFrame separates task-level planning from frame-level Minecraft control. A vision-language planner decomposes an instruction into ordered goals and caches each goal’s embedding for a separate controller, which uses current visual input to choose keyboard, mouse-button, and camera actions. Across four controlled kill tasks, it averages 51% success versus JARVIS-VLA-7B’s 31%; on mining, it scores 96% versus 100%.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.