Reproduce “Weasel: Out-of-Domain Generalization for Web Agents via Importance-Diversity Data Selection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fatemeh Pesaran Zadeh, Seyeon Choi, Xing Han Lù, Siva Reddy, Gunhee Kim
WEASEL improves web-agent transfer to unseen websites by selecting trajectory steps that are both goal-relevant and behaviorally diverse. Combined with target-centered accessibility-tree pruning and model-compatible reasoning synthesis, it reduces fine-tuning time by roughly 10–12× while improving zero-shot success on WebArena-Lite, WebArena, and WorkArena.
Key insights
- Relevance and diversity are complementary: on WebArena-Lite, combining them reaches 14.5% success for Qwen2.5-7B, compared with 10.9% using importance alone and 7.9% using diversity alone.
- Differences in model answers—reasoning plus actions—provide particularly useful diversity signals, while webpage-state diversity adds coverage of distinct browser contexts.
- Compact, locally focused supervision can improve transfer more effectively than simply training on a larger unfiltered dataset, although complex WorkArena tasks remain difficult.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.