Reproduce “Knowledge is a Region in Weight Space for Fine-tuned Language Models” on OpenResearch

About this paper

Almog Gueta, Elad Venezian, Colin Raffel, Noam Slonim, Yoav Katz, Leshem Choshen

Researchers at IBM Research, Technion, and UNC Chapel Hill demonstrate that finetuned language models converge to specific 'knowledge regions' in weight space, rather than isolated points. They show that models sampled from the convex hull of these regions often exhibit higher performance than individual finetuned models, providing a theoretical basis for the empirical success of model averaging. Furthermore, using the centroid of these regions as a finetuning initialization leads to improved performance, especially in few-shot learning scenarios.

Key insights

• Knowledge in finetuned language models is encoded not as isolated points, but within robust, connected 'knowledge regions' (convex hulls) in weight space.
• The 'center' or interior of these knowledge regions often yields models with better performance than the individual models found at the 'edges' by standard finetuning.
• The movement in weight space during finetuning is primarily determined by the type of data, leading models trained on similar data or tasks to cluster together.

Summary by alphaXiv.