Reproduce “AnyStyle: A Single LoRA is Sufficient for Image-Guided Style Transfer” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yongwen Lai, Chaoqun Wang

AnyStyle introduces a framework for image-guided style transfer that uses a single LoRA for style alignment and training-free attention modulation for content preservation. This approach improves upon dual-LoRA methods, achieving higher prompt and style alignment metrics (e.g., CLIP-T: 32.5573, CLIP-S: 67.9291) while maintaining strong content fidelity and enhancing control over content-style balance.

Key insights

  • Dual-LoRA architectures for style transfer often suffer from feature entanglement, making it challenging to achieve a desired balance between content and style.
  • Training-free structural guidance using the internal attention mechanisms of pre-trained models is more effective and efficient for content preservation than a dedicated content LoRA.
  • Applying a single LoRA to specific, later transformer blocks of a Diffusion Transformer (DiT) is sufficient for robust style alignment.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.