Reproduce “Enhancing Intent Understanding for Ambiguous prompt: A Human-Machine Co-Adaption Strategy” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yangfan He, Jianhui Wang, Yijin Wang, Yan Zhong, Xinyuan Song, Junjiang Lin, Xinhang Yuan, Jingqun Tang and 13 more
A human-machine co-adaptation strategy was developed to improve how text-to-image models interpret ambiguous user prompts, utilizing mutual information maximization and reinforcement learning. The system significantly enhances user satisfaction and the semantic alignment of generated images, streamlining the creative process for non-expert users.
Key insights
- Maximizing mutual information between user intent and generated images, effectively approximated by CLIP scores, serves as a robust objective for human-machine co-adaptation.
- Reinforcement learning, specifically PPO, is crucial for enabling the model to dynamically learn from user feedback, leading to a significant reduction in interaction rounds.
- Discrete editing operations on prompt elements and attention maps provide intuitive, fine-grained control, allowing the system to adaptively align with complex or ambiguous user intentions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.