Reproduce “TempJail: Temporal Jailbreak Attacks against Image-to-Video Generation Models” on OpenResearch

About this paper

Qi Lu, Zehui Guo, David Yuanda Gan, Zijing Li, Hengda Zhang, Weijun Xu, Qiankun Zhang

TempJail is a framework designed to exploit temporal vulnerabilities in image-to-video (I2V) generation models by subtly decoupling malicious content into camouflaged image and text inputs. It successfully generates videos with unsafe semantics that evolve over time, bypassing current safety filters in commercial I2V models like Kling and PixVerse, achieving average attack success rates up to 65.54%.

Key insights

• Image-to-video (I2V) models possess a unique 'temporal vulnerability' where unsafe content can emerge dynamically through coherent composition over time, even when individual inputs appear benign, thereby bypassing current static-focused safety filters.
• Effectively bypassing I2V safety filters requires a multi-modal camouflage strategy that abstracts complex malicious narratives and subtly embeds them across both visual and textual input components.
• The inherent capacity of I2V models for autonomous temporal planning, designed for high-quality generation and temporal coherence, can be exploited to orchestrate the progressive manifestation of malicious content.

Summary by alphaXiv.