Reproduce “A Theoretical Framework for LLM Fine-tuning Using Early Stopping for Non-random Initialization” on OpenResearch

About this paper

Zexuan Sun, Garvesh Raskutti

Researchers at the University of Wisconsin, Madison, developed a statistical framework integrating early stopping theory with the Neural Tangent Kernel (NTK) for transformer-based large language models. The framework formally explains why few fine-tuning epochs are sufficient, providing convergence guarantees and elucidating task arithmetic, with empirical validation on GPT-Neo-1.3B models showing a link between kernel eigenvalue decay and sample efficiency.

Key insights

• Fine-tuned transformer networks can be linearized around their pretrained state, with the L2-norm of the approximation error bounded by O(d_l^(-1/2)), indicating NTK stability during fine-tuning from pretrained weights.
• The effectiveness of early stopping in fine-tuning is formally explained by the optimal stopping time being constrained by the initial closeness of the pretrained model to the target function and the controlled accumulation of errors.
• The research established a formal link between the convergence rate with respect to sample size and the eigenvalue decay rate of the empirical kernel matrix, providing a metric for 'task-model compatibility' to predict sample efficiency.

Summary by alphaXiv.