Reproduce “Understanding and Minimising Outlier Features in Neural Network Training” on OpenResearch

About this paper

Bobby He, Lorenzo Noci, Daniele Paliotta, Imanol Schlag, Thomas Hofmann

The paper from ETH Zürich identifies the interplay of normalization layers and optimizer adaptivity as causes of Outlier Features (OFs) in Transformer training. It introduces a novel Outlier Protected (OP) block and demonstrates that non-diagonal preconditioners in optimizers effectively minimize OFs, leading to up to 4 orders of magnitude lower Kurtosis and substantially improved 8-bit quantization for models like OPT-125m with only a 0.16 perplexity increase.

Key insights

• Outlier Features are a pervasive phenomenon in standard Transformer training, arising from a complex interplay of normalization layer design and optimizer adaptivity, rather than isolated factors.
• There is a strong empirical correlation between the emergence of Outlier Features and poor signal propagation dynamics during training, where higher OFs correspond to increased input-wise correlations.
• Non-diagonal preconditioners in optimizers effectively mitigate the sub-leading order (quadratic) terms in activation moment updates, which are identified as the primary drivers of Kurtosis increase and OF emergence.

Summary by alphaXiv.