Reproduce “Lightspeed Geometric Dataset Distance via Sliced Optimal Transport” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Khai Nguyen, Hai Nguyen, Tuan Pham, Nhat Ho
A new method, sliced optimal transport dataset distance (s-OTDD), provides an efficient and geometrically meaningful way to compare machine learning datasets by leveraging sliced optimal transport. It achieves near-linear computational complexity, drastically reducing processing time, and exhibits strong correlations with exact optimal transport dataset distance while effectively predicting transferability and guiding data augmentation strategies.
Key insights
- The Moment Transform Projection (MTP) is injective under certain regularity conditions, allowing it to meaningfully map distributions over features to scalar values for slicing.
- s-OTDD satisfies the properties of a mathematical metric on the space of distributions over feature-label pairs, providing strong theoretical guarantees.
- By leveraging random 1D projections, s-OTDD achieves (near-)linear computational complexity (O(L(n log n + dn))), scaling efficiently with dataset size and feature dimensions while being independent of the number of classes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.