Reproduce “Venus: Benchmarking and Empowering Multimodal Large Language Models for Aesthetic Guidance and Cropping” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tianxiang Du, Hulingxiao He, Yuxin Peng

Peking University researchers introduced Venus, a two-stage framework and the AesGuide dataset, to equip multimodal large language models (MLLMs) with aesthetic guidance and interpretable cropping abilities. The framework achieved state-of-the-art aesthetic cropping performance on the FLMS benchmark, with Venus-Q demonstrating an 87.01% IoU, a +1.50 IoU improvement over prior methods, while also providing superior aesthetic feedback compared to commercial MLLMs.

Key insights

  • A progressive training approach, mimicking human aesthetic reasoning from overall impressions to detailed analysis and actionable guidance, is crucial for building robust aesthetic guidance capabilities in MLLMs.
  • Developing strong aesthetic guidance capabilities in MLLMs (Stage 1) is fundamental and transferable, significantly enhancing their performance in subsequent aesthetic cropping tasks (Stage 2).
  • Integrating explicit CoT-based aesthetic rationales during the training of cropping models not only improves cropping accuracy but also imbues the models with interpretability and interactivity, addressing limitations of prior black-box approaches.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.