Reproduce “Group Distributionally Robust Dataset Distillation with Risk Minimization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Saeed Vahidian, Mingyu Wang, Jianyang Gu, Vyacheslav Kungurtsev, Wei Jiang, Yiran Chen

A "Double-DRO-based DD procedure" enhances dataset distillation by promoting group robustness and within-group generalization through the integration of Distributionally Robust Optimization. This method consistently improves Top-1 test accuracy across standard benchmarks, notably gaining 3.1% on ImageNet-10, and significantly mitigates performance drops for underrepresented subgroups and in domain-shifted environments.

Key insights

  • Integrating Distributionally Robust Optimization (DRO) principles directly into the dataset distillation process significantly enhances model generalization and robustness to subgroup shifts.
  • Clustering real data points based on their proximity to synthetic samples allows for explicit identification and robust optimization across diverse subgroups.
  • Utilizing Conditional Value at Risk (CVaR) as a loss aggregation mechanism within clusters effectively approximates a local DRO problem, improving within-group generalization by targeting worst-case loss scenarios.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.