Reproduce “EditGarment: An Instruction-Based Garment Editing Dataset Constructed with Automated MLLM Synthesis and Semantic-Aware Evaluation” on OpenResearch

About this paper

Deqiang Yin, Junyi Guo, Huanda Lu, Fangyu Wu, Dongming Lu

The first large-scale, instruction-based dataset for standalone garment image editing, EditGarment, was created using an automated MLLM-based pipeline and a novel semantic-aware evaluation metric, FEditScore. Models fine-tuned on this dataset achieve higher editing quality and semantic consistency compared to general image editing models, enabling precise modifications based on natural language instructions.

Key insights

• Automated dataset generation for specialized domains like garment editing is feasible with MLLMs but requires domain-specific categorization of edits and a robust, semantic-aware evaluation framework to ensure quality and diversity.
• The FEditScore, which leverages a semantic dependency graph and weighted 'Yes/No' questions (Instruction-Critical, Instruction-Dependent, Context-Preserving), provides a more effective and interpretable metric for evaluating fine-grained edits than traditional image quality metrics.
• Categorizing garment modifications into predefined types (e.g., structural, material, color alterations) helps mitigate biases in MLLMs and ensures a balanced distribution of diverse and complex editing instructions, addressing the long-tail attribute problem.

Summary by alphaXiv.