Reproduce “Measuring Dataset Granularity” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yin Cui, Zeqi Gu, Dhruv Mahajan, Laurens van der Maaten, Serge Belongie, Ser-Nam Lim
Researchers from Cornell University, Cornell Tech, and Facebook AI introduce the first quantitative framework for measuring dataset granularity, establishing a continuous scale to quantify the difficulty of distinctions within any dataset. Their proposed RankM measure effectively correlates with classification accuracy (\u03c1 \u2248 0.998), transfer learning performance (\u03c1 \u2248 0.980), few-shot learning challenges, and adversarial robustness, revealing that granularity varies significantly even within purportedly fine-grained datasets.
Key insights
- Dataset granularity is a continuous scale, not a binary property, and can be quantitatively measured based on sample similarity within a feature space.
- Granularity varies significantly even within datasets typically labeled 'fine-grained', with subsets exhibiting vastly different levels of distinction difficulty ('bitter' vs. 'sweet' tasters).
- The perceived granularity of a dataset can be influenced by the feature representation used, such as features extracted from different pre-trained neural networks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.