Reproduce “When Minor Edits Matter: LLM-Driven Prompt Attack for Medical VLM Robustness in Ultrasound” on OpenResearch

About this paper

Yasamin Medghalchi, Milad Yazdani, Amirhossein Dabiriaghdam, Moein Heidari, Mojan Izadkhah, Zahra Kavian, Giuseppe Carenini, Lele Wang and 2 more

Researchers at the University of British Columbia developed an LLM-driven framework to evaluate the robustness of Medical Vision-Language Models (Med-VLMs) in ultrasound imaging against subtle prompt modifications. This framework demonstrated that Med-VLMs experienced an accuracy drop of 14.89% to 26.08% when attacked, primarily affecting low-confidence predictions with only 2-3 minimal textual edits.

Key insights

• Med-VLMs are substantially vulnerable to subtle, LLM-generated prompt attacks, even without access to internal model parameters, raising concerns for their safe deployment in clinical settings.
• The success of an attack is strongly correlated with the Med-VLM's initial confidence (logit margin) in the correct answer, indicating that low-confidence predictions are more susceptible to manipulation.
• While smaller attacker LLMs can be effective at inducing failures, larger LLMs tend to produce more naturalistic and semantically preserved adversarial edits.

Summary by alphaXiv.