Reproduce “Image Generators with Conditionally-Independent Pixel Synthesis” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ivan Anokhin, Kirill Demochkin, Taras Khakhulin, Gleb Sterkin, Victor Lempitsky, Denis Korzhenkov

Conditionally-Independent Pixel Synthesis (CIPS) generates high-quality images by computing each pixel's color independently using a coordinate-based Multi-Layer Perceptron (MLP) and a shared latent vector, demonstrating that spatial convolutions are not essential for state-of-the-art image synthesis. The architecture achieves FID scores competitive with StyleGANv2 and exhibits superior spectral properties, along with unique capabilities like arbitrary resolution and non-Cartesian synthesis.

Key insights

  • High-quality image generation is achievable without explicit spatial convolutions, challenging a long-standing assumption in the field.
  • Sophisticated positional encodings, combining Fourier features and learnable coordinate embeddings, are crucial for enabling coordinate-based MLPs to capture high-frequency details.
  • CIPS generators produce images with superior Fourier spectral properties, aligning more closely with real image statistics than traditional convolutional generators.
  • The two types of positional encodings specialize in different frequency ranges, with Fourier features handling low frequencies and coordinate embeddings capturing high-frequency details and semantic information.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.