Reproduce “Low-Rank Curvature for Zeroth-Order Optimization in LLM Fine-Tuning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hyunseok Seung, Jaewoo Lee, Hyunsuk Ko
LOREN, a zeroth-order optimization method, addresses challenges in large language model fine-tuning by integrating low-rank curvature adaptation and variance reduction. This approach yields higher accuracy, faster convergence, and reduced memory footprint compared to previous zeroth-order techniques across various LLMs and benchmarks.
Key insights
- Reframing ZO gradient preconditioning as optimizing an adaptive perturbation distribution within NES allows for dynamic curvature learning.
- A memory-efficient, Kronecker-factored rank-1 block-diagonal approximation can effectively model and capture the anisotropic curvature of LLM loss landscapes in a ZO setting.
- The RLOO estimator effectively reduces the variance of ZO gradient estimates by utilizing multiple samples per batch, improving stability and convergence without significant memory increase.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.