Reproduce “Instruction Anchor: Dissecting the Mechanistic Dynamics of Modality Arbitration” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yu Zhang, Mufan Xu, Xuefeng Bai, Kehai Chen, Pengfei Zhang, Yang Xiang, Min Zhang

The paper examines how multimodal models follow instructions about trusting image or text when they conflict. Tests on Qwen2.5-VL-7B and InternVL3-8B use attention knockouts and readouts to trace information flow. Instruction-token readouts rise from near chance in shallow layers to above 95% in deep layers. Blocking selected heads disrupts following, but does not establish an exclusive route or general reliability.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.