Reproduce “On the Adversarial Robustness of Multi-Modal Foundation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Christian Schlarmann, Matthias Hein
Researchers from the Tübingen AI Center assessed the adversarial robustness of OpenFlamingo, a multi-modal foundation model, revealing that imperceptible visual perturbations can drastically alter its textual outputs. The study showed targeted attacks can achieve 100% success for specific phrases at an -norm of 4/255, while untargeted attacks reduced image captioning quality (CIDEr scores dropped from 84.01 to 1.69 on COCO) with similarly subtle modifications.
Key insights
- Multi-modal foundation models, such as OpenFlamingo, are highly susceptible to adversarial visual attacks that involve imperceptible pixel-level perturbations.
- Adversaries can precisely control the model's textual outputs through targeted attacks, forcing it to generate specific, potentially malicious, content.
- Neither few-shot inference nor partial perturbation application significantly enhances the model's robustness against these types of attacks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.