Reproduce “Zeta: Dual Whitening for Matrix Optimization via Coordinate-Adaptive Preconditioning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Kaiwen Chen, Shuhai Zhang, Zimo Liu, Linxiao Li, Ying Sun, Yuchen Li, Yifan Zhang, Bo Han and 2 more

Zeta combines coordinate-wise normalization with matrix-level orthogonalization for neural-network training. It divides momentum entries by their running second moments before applying Newton–Schulz iterations to matrix-shaped weights. On Qwen3-1.7B, Zeta reaches the same training loss 1.67× faster by iteration count and 1.64× faster by elapsed time than AdamW, and reports the best average accuracy across 12 benchmarks in tested settings, not every task.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.