Reproduce “How Data Augmentation Shapes Neural Representations” on OpenResearch

About this paper

Tianxiao He, Alex H. Williams, Sarah E. Harvey

The authors compare hidden-layer responses to clean images across models trained with different augmentation settings. They align response patterns across rotations and reflections to measure distances. In same-seed ResNet-18 experiments, aligned representations form ordered patterns across augmentation strengths, unlike unaligned ones. Greater divergence between augmentation trajectories is associated with larger ensemble gains in these experiments, but causation is not established.

Key insights

• Data augmentation (DA) consistently sculpts neural representations along structured, well-behaved trajectories within the proposed Riemannian shape space, a phenomenon not observed in naive Euclidean space.
• The geometric divergence between representations induced by different DA types, measured by geodesic angles, is predictive of functional improvements when those models are ensembled.
• Data augmentation leads to layer-dependent landmark distortions, where specific probe images are differentially 'contracted' or 'expanded' in the representation space based on the DA type.

Summary by alphaXiv.