Reproduce “Adversarial attacks against Modern Vision-Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alejandro Paredes La Torre
This research systematically assessed the adversarial robustness of open-source Vision-Language Model agents in an interactive e-commerce simulation, revealing that LLaVA-v1.5-7B is highly susceptible to white-box gradient-based attacks (up to 66.9% ASR) while Qwen2.5-VL-7B demonstrates significantly greater resilience, maintaining near-clean operational performance.
Key insights
- Open-source VLMs, specifically LLaVA-v1.5-7B, are highly vulnerable to relatively simple white-box gradient-based adversarial attacks in a realistic interactive context.
- There is a substantial difference in adversarial resilience between open-source VLM families (e.g., LLaVA vs. Qwen), suggesting architectural or training variations influence robustness.
- CLIP-based spectral attacks are a particularly potent attack vector, consistently achieving high success rates by disrupting feature-space geometry in CLIP embeddings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.