Reproduce “Beyond the noise: intrinsic dimension estimation with optimal neighbourhood identification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Antonio Di Noia, Iuri Macocco, Aldo Glielmo, Alessandro Laio, Antonietta Mira
Researchers from ETH Zurich and Università della Svizzera italiana developed the Adaptive Binomial Intrinsic Dimension Estimator (ABIDE), an iterative framework that automatically identifies optimal local scales for intrinsic dimension (ID) estimation. This method robustly addresses noise and local density variations in high-dimensional data, yielding more accurate and statistically validated ID estimates than non-adaptive approaches across synthetic and real-world datasets.
Key insights
- Robust and automatic intrinsic dimension estimation is achievable by iteratively and jointly optimizing the ID estimate with point-wise adaptive neighborhood selection.
- Explicitly enforcing the locally constant density assumption within neighborhoods, through a likelihood-ratio test, is crucial for distinguishing true manifold dimensions from noise and local density variations.
- Providing theoretical guarantees (numerical convergence, consistency, asymptotic normality) and a goodness-of-fit test for model validation enhances the statistical reliability and trustworthiness of ID estimates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.