Reproduce “Decoupled Guidance: Disentangling Subject and Context Pathways in Text-to-Image Personalization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Seongmin Kim, Kyucheol Shin, Heesun Jung, Jinseo Kim, Sungyong Baik

Decoupled Guidance (DeGu) introduces a framework to disentangle subject identity and scene context pathways in text-to-image personalization, addressing the fidelity-editability trade-off. It consistently improved subject fidelity and text editability across various generative models, including SD v2.1, SDXL, and SD v3.5 DiT architectures, with user studies confirming higher preference for generated images.

Key insights

  • "Conditioning entanglement" and "attention collapse" are fundamental structural bottlenecks in existing personalization methods, leading to competition for attention resources rather than merely being a capacity limitation.
  • Increasing the number of learnable subject tokens intensifies attention collapse and worsens the fidelity-editability trade-off, challenging the hypothesis that insufficient capacity is the primary cause.
  • Spatially partitioning fidelity and editability guidance at inference time, using internally derived cross-attention masks, is crucial for preventing reference background leakage and achieving an optimal balance between subject preservation and context adherence.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.