Reproduce “Data Selection Through Iterative Self-Filtering for Vision-Language Settings” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Andrei Liviu Nicolicioiu, Sarvjeet Singh Ghotra, Morgane M. Moss, Aaron Courville

Mila researchers introduce Self-Filtering, an iterative, bootstrapped data filtering method for vision-language models that autonomously curates high-quality datasets. The approach enhances downstream performance without relying on external pre-trained models or additional curated data, achieving results comparable to or surpassing those obtained with powerful existing filters.

Key insights

  • Effective data filtering can be achieved intrinsically, without external pre-trained models, demonstrating a self-improving loop where better models curate better datasets that, in turn, train better models.
  • Balancing the exploitation of current model knowledge (oversampling 'likely clean' data) with exploration of the full data distribution (retaining all samples) is crucial for sustained generalization.
  • The primary benefit of Self-Filtering stems from creating a higher-quality static dataset, rather than a dynamic curriculum effect from the iterative training, though continuous training offers practical advantages.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.