Reproduce “CareCom: Generative Image Composition with Calibrated Reference Features” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiaxuan Chen, Bo Zhang, Qingdong He, Jinlong Peng, Li Niu
CareCom introduces a multi-reference generative image composition framework that calibrates global and local foreground features to enhance detail preservation and pose/view adjustment. The model achieved a leading FOSScore of 0.883 and Quality Score of 47.07 on the MureCom dataset, surpassing existing generative composition methods in overall quality and contextual compatibility while maintaining competitive fidelity.
Key insights
- Leveraging multiple reference images significantly enhances a model's ability to preserve foreground details and adjust its pose/view for seamless integration into diverse backgrounds.
- Calibrating reference features (global for pose/view, local for details) to match ground-truth features that are contextually compatible is more efficient than generating entirely new reference images.
- Combining both uncalibrated (for core object details) and calibrated (for background alignment) features within the generative process yields superior compositional results.
- The proposed feature calibration modules effectively guide the diffusion model to synthesize foregrounds that are both faithful to the reference and harmonized with the background.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.