Reproduce “Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinbei Ma, Yiting Wang, Yao Yao, Tongxin Yuan, Aston Zhang, Zhuosheng Zhang, Hai Zhao
Researchers at Shanghai Jiao Tong University and Meta empirically demonstrate that multimodal large language model agents are consistently susceptible to environmental distractions in GUI environments, leading to unfaithful actions and compromising user goals. This vulnerability extends to advanced models like GPT-4o and can be exploited through subtle environment injection.
Key insights
- Increased perception capabilities in MLLMs do not inherently guarantee faithfulness and can, in some cases, increase susceptibility to distractions.
- The method of modality fusion, particularly textual augmentation of visual inputs, needs careful consideration as it can sometimes exacerbate distraction risks.
- A robust agent requires not just strong perception but also sophisticated cross-modal reasoning to differentiate relevant goal-oriented content from environmental noise and understand UI hierarchical constraints.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.