Reproduce “Affective Image Editing: Shaping Emotional Factors via Text Descriptions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Peixuan Zhang, Shuchen Weng, Chengxuan Zhu, Binghao Tang, Zijian Jia, Si Li, Boxin Shi
A method called AIEdiT enables comprehensive affective image editing by adaptively shaping multiple emotional factors throughout an image, guided by fine-grained textual instructions. This approach, which includes the new EmoTIPS dataset, achieved leading performance against state-of-the-art methods across FID (27.93), Sem-C (0.685), KLD (2.4373), and user preference (39.68%).
Key insights
- Representing universal emotional priors in a continuous spectrum, learned via contrastive methods, effectively captures nuanced emotional requests from text descriptions.
- An emotional mapper, built with multi-modal transformers and key semantic features, effectively translates abstract emotional requests into concrete visual semantic representations for image manipulation.
- Leveraging MLLMs as supervisors through a sentiment alignment loss during training significantly improves the model's ability to shape emotional factors accurately and ensures emotional consistency in generated images.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.