Reproduce “In-Context Brush: Zero-shot Customized Subject Insertion with Context-Aware Latent Space Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yu Xu, Fan Tang, You Wu, Lin Gao, Oliver Deussen, Hongbin Yan, Jintao Li, Juan Cao and 1 more
In-Context Brush introduces a training-free method for inserting a reference subject into a masked region of a target image while retaining its identity, following text-based edits, and matching the surrounding scene. By manipulating prompt- and subject-related latent features within Flux’s multimodal diffusion transformer, the method achieves a FID of 122.61, CLIP-I scores of 0.7957/0.7924 for insertion/editing, and user-study preference rates of 65.95% for identity alignment and 73.00% for editing alignment.
Key insights
- Prompt controllability and subject fidelity can be adjusted directly through separate latent-shift coefficients, avoiding parameter updates or subject-specific embeddings.
- Attention heads provide a useful basis for selective prompt amplification, helping suppress reference attributes that conflict with the requested edit.
- Maintaining the original background throughout denoising is important: removing token blending raises FID from 122.61 to 129.47, while removing head activation raises it to 123.82.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.