Reproduce “Untwisting RoPE: Frequency Control for Shared Attention in DiTs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aryan Mikaeili, Or Patashnik, Andrea Tagliasacchi, Daniel Cohen-Or, Ali Mahdavi-Amiri
Analyzing Rotary Positional Embeddings (RoPE) in Diffusion Transformers (DiTs), this work identifies that high-frequency components are responsible for 'reference copying' during shared attention. It introduces a frequency-aware modulation method for RoPE that enables controllable style-aligned image generation, preventing unintended content duplication from reference images.
Key insights
- High-frequency components of RoPE are strongly positionally sensitive and are the primary driver of 'reference copying' by biasing attention towards spatially aligned tokens.
- Low-frequency RoPE components promote global semantic similarity and are less sensitive to precise positional shifts, contributing to overall structural coherence.
- Targeted modulation of RoPE's frequency bands on reference keys can effectively rebalance positional bias, shifting attention from content-copying towards desired semantic style transfer.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.