Reproduce “Generalization Bounds for Rank-sparse Neural Networks” on OpenResearch

About this paper

Antoine Ledent, Rodrigo Alves, Yunwen Lei

Princeton researchers developed GOEDEL-ARCHITECT, an agentic framework for formal theorem proving in Lean 4 that employs blueprint generation and refinement. The system achieves state-of-the-art performance, including 100% on MiniF2F-test and 75.6% pass@1 on PutnamBench, while leveraging an open-weight model and reducing computational costs by up to 500 times compared to existing pipelines.

Key insights

• Schatten p quasi-norms effectively quantify rank sparsity, enabling generalization bounds to seamlessly interpolate between exact rank-based and magnitude-based regularization.
• The parametric interpolation technique provides a more nuanced and adaptive measure of network capacity, bridging the gap between traditional parameter-counting and norm-based generalization regimes.
• Explicitly incorporating low-rank structures in generalization bounds offers theoretical justification for why overparameterized networks can generalize well, linking to the observed phenomenon of neural rank collapse.

Summary by alphaXiv.