Reproduce “S2WAT: Image Style Transfer via Hierarchical Vision Transformer using Strips Window Attention” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chiyu Zhang, Xiaogang Xu, Lei Wang, Zaiyan Dai, Jun Yang

The S2WAT framework introduces a hierarchical vision transformer for image style transfer, leveraging a novel Strips Window Attention and an adaptive 'Attn Merge' module to capture both local and global dependencies. This approach generates high-quality stylized images that preserve content details and embody artistic styles while effectively mitigating artifacts and content leakage.

Key insights

  • The 'locality problem' in window-based attention, caused by its inherently localized nature, is a primary source of grid-like artifacts in generative vision transformer applications like style transfer.
  • Combining diverse window attention shapes (horizontal, vertical, and square strips) within a single module can effectively expand the receptive field and mitigate artifacts without requiring deep stacks of shifted windows.
  • An adaptive fusion mechanism, rather than static methods like summation or concatenation, is crucial for harmoniously integrating multi-scale features from different attention types, preventing information loss and ensuring smooth stylization.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.