Reproduce “T2I-ConBench: Text-to-Image Benchmark for Continual Post-training” on OpenResearch

About this paper

Zhehao Huang, Yuhang Liu, Yixin Lou, Zhengbao He, Mingzhen He, Wenxing Zhou, Tao Li, Kehan Li and 2 more

Researchers from Shanghai Jiao Tong University and Huawei introduce T2I-ConBench, a comprehensive benchmark for evaluating continual post-training methods in text-to-image diffusion models, revealing that while LoRA variants minimize catastrophic forgetting, they struggle with item customization tasks, and that cross-task generalization remains a significant challenge with most methods falling short of joint training approaches across realistic scenarios involving item customization and domain enhancement.

Key insights

• Different continual learning methods excel in different aspects with no single approach dominating all evaluation dimensions
• Joint training on all data simultaneously is not always optimal and can struggle with conflicting multi-task optimization demands
• LoRA-based parameter isolation methods effectively prevent forgetting but at the cost of reduced adaptation capability
• Cross-task generalization requires active integration of prior representations with new information rather than simple parameter isolation

Summary by alphaXiv.