Reproduce “Learning to See by Looking at Noise” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Manel Baradad, Jonas Wulff, Tongzhou Wang, Phillip Isola, Antonio Torralba

Researchers from MIT CSAIL demonstrated that visual representations can be learned effectively from procedurally generated noise, completely bypassing the need for real image datasets. Their method, using various noise models including untrained StyleGANs and statistical image models, achieved up to 40.06% top-1 accuracy on ImageNet-100 with a ResNet50 encoder, indicating that naturalistic structural properties and dataset diversity are key to effective learning.

Key insights

  • Visual representations can be learned with surprising effectiveness solely from procedurally generated noise, achieving non-trivial performance on various downstream vision tasks without any exposure to real-world images.
  • The critical factor for effective learning from synthetic data is not photorealism, but rather the 'naturalism' of structural properties, such as mimicking natural image spectrum, color profiles, oriented structures, and occlusion patterns.
  • Dataset-level diversity (high recall in feature space) is more important than strict realism (high precision) for developing robust and transferable visual features from synthetic data, with combinations of different noise types enhancing overall performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.