Reproduce “Generalization Bounds with Data-dependent Fractal Dimensions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Benjamin Dupuis, George Deligiannidis, Umut Şimşekli
This research develops new generalization bounds for deep learning models by introducing a data-dependent pseudo-metric and associated fractal dimension, which removes the restrictive Lipschitz continuity assumption prevalent in prior work. Experiments show this dimension strongly correlates with generalization error, demonstrating improved predictive power over Euclidean-based dimensions and facilitating its numerical estimation via persistent homology.
Key insights
- Generalization behavior can be effectively bounded by complexity measures defined in the loss space via a data-dependent pseudo-metric, allowing for bounds that are applicable to non-Lipschitz loss functions common in deep learning.
- The concept of 'geometric stability' of the hypothesis set to data perturbations simplifies the mutual information term in generalization bounds for random hypothesis spaces, providing a more interpretable framework.
- Persistent homology offers a robust and computationally feasible method for estimating data-dependent fractal dimensions, bridging theoretical generalization bounds with practical numerical analysis.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.