Reproduce “TeachAugment: Data Augmentation Optimization Using Teacher Knowledge” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Teppei Suzuki

Denso IT Laboratory developed TeachAugment, an online data augmentation method utilizing a teacher model and a neural network-based augmenter to generate challenging yet semantically coherent images for model training. This approach achieved a 22.2% top-1 error on ImageNet with ResNet-50 and consistently improved performance across classification, segmentation, and self-supervised learning tasks without requiring hyperparameter tuning.

Key insights

  • The teacher model effectively prevents the augmentation process from collapsing into meaningless deformations by simultaneously minimizing the teacher's loss on augmented images.
  • An Exponential Moving Average (EMA) teacher is superior to a pretrained teacher, adapting during training and simplifying the overall architecture.
  • A differentiable, neural network-based augmentation model offers flexibility and expressiveness, simplifying the search space and enabling efficient gradient-based optimization.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.