Reproduce “Training-and-Prompt-Free General Painterly Harmonization via Zero-Shot Disentenglement on Style and Content References” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Teng-Fang Hsiao, Bo-Kai Ruan, Hong-Han Shuai
Researchers from National Yang Ming Chiao Tung University present TF-GPH, a training-and-prompt-free method for general painterly harmonization that leverages zero-shot disentanglement within a Stable Diffusion model. This approach achieves state-of-the-art results across various benchmarks by minimizing content disruption and offering flexible control over stylization.
Key insights
- Precise manipulation of a diffusion model's attention mechanism, particularly through a Similarity Disentangle Mask, can effectively manage feature flow from multiple distinct image inputs (content, style, composite).
- Similarity Reweighting provides a fine-grained, inference-time adjustable control over the balance between stylization strength and content preservation, mitigating content disruption common in other methods.
- A truly training-free and prompt-free approach to general painterly harmonization is achievable, offering high adaptability and generalizability to diverse and unseen artistic styles and content.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.