Reproduce “Task2Vec: Task Embedding for Meta-Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless Fowlkes, Stefano Soatto, Pietro Perona
TASK2VEC proposes a method to create fixed-dimensional vectorial representations of machine learning tasks using the Fisher Information Matrix of a probe network. These task embeddings enable efficient meta-learning, showing strong correlation with semantic and taxonomic task similarities and demonstrating substantial improvements in selecting optimal pre-trained models for new tasks, particularly in low-data environments.
Key insights
- The Fisher Information Matrix, when applied to a probe network's features, effectively quantifies the 'informative content' of a task, leading to embeddings that capture semantic and taxonomic similarities.
- The L1 norm of the TASK2VEC embedding correlates positively with the perceived difficulty or complexity of a task for a given probe network.
- The task embeddings are invariant to label space and encode task-specific domain information by focusing on data points near decision boundaries, unlike simpler domain-only embeddings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.