Reproduce “Spectral Reach: Understanding Neural Scaling as Progress into the Spectral Tail” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Konstantin Nikolaou, Jonas Scheunemann, Sven Krippendorf, Samuel Tovey, Christian Holm

The paper introduces a gradient-based diagnostic for tracking where learning is focused in the empirical neural tangent kernel spectrum without constructing the full matrix. It defines spectral position, which falls as training shifts toward weaker modes, and reaches lower values in larger models. In reported experiments, analysis adds less than 2× compute and memory, but the causal mechanism remains unresolved.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.