Reproduce “Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Guilin Liu, Rohan Taori, Ting-Chun Wang, Zhiding Yu, Shiqiu Liu, Fitsum A. Reda, Karan Sapra, Andrew Tao and 1 more

Transposer introduces a deep learning framework for universal texture synthesis that reinterprets transposed convolution, dynamically generating filters from input feature maps guided by self-similarity. This enables the system to generalize to unseen textures with near real-time performance, synthesizing 256x256 images in 43ms while preserving complex structural patterns and outperforming previous methods in fidelity.

Key insights

  • Texture synthesis operations, traditionally involving shifting and assembling, can be reformulated as a transposed convolution using dynamically generated, content-aware filters derived from the input feature map itself.
  • Self-similarity maps, computed from the auto-correlation of input features at multiple scales, effectively capture and encode the repetitive structural patterns essential for coherent texture reproduction.
  • A multi-scale UNet-like architecture, combined with a balanced set of perceptual, style, and GAN losses, enables the framework to preserve both fine-grained details and larger structural elements during synthesis.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.