Reproduce “Variational Task Vector Composition” on OpenResearch

About this paper

Boyuan Zhang, Yingjun Du, Xiantong Zhen, Ling Shao

This research introduces Variational Task Vector Composition, a Bayesian inference framework that enables sample-specific, uncertainty-aware, and sparse knowledge integration in deep learning models. It consistently improves image classification accuracy by adapting composition coefficients to individual inputs and selectively activating relevant task components, achieving robust performance across various datasets and Vision Transformer architectures.

Key insights

• Sample-specific adaptation of task vector composition coefficients is crucial for achieving robust performance, especially on datasets with high internal diversity or significant domain differences.
• Explicitly modeling uncertainty through a variational inference framework enhances model robustness and generalization, particularly in the presence of cross-domain variations.
• Spike-and-Slab priors effectively reduce structural redundancy in task vector spaces, leading to more efficient, sparse, and interpretable knowledge integration without sacrificing accuracy.

Summary by alphaXiv.