Reproduce “A Good Initialization is All You Need for Faithful Visual Attribution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zihan Gu, Jiayu Wang, Hua Zhang, Yue Hu
Researchers from the Institute of Information Engineering, Chinese Academy of Sciences, and the University of Chinese Academy of Sciences developed COPAIR and TRACE, two methods to generate compact, fixed-size top-k visual attribution masks. These methods enhance explanation faithfulness and efficiency in ImageNet classification and significantly improve MLLM hallucination attribution and repair, achieving repair rates of up to 96% with single interventions.
Key insights
- Directly optimizing for compact, fixed-size k-region masks (termed 'mask-first' attribution) provides more actionable and relevant visual explanations for practical AI applications, particularly for targeted interventions.
- Intelligent initialization strategies, like COPAIR's coarse-spatial pairing or TRACE's cross-entropy-based sampling, can effectively guide and improve the performance and efficiency of existing full-ordering search algorithms by shifting the attribution search frontier.
- The cross-entropy method, when adapted for discrete k-hot space, is effective in discovering optimal combinations of interacting fine regions, leading to high-fidelity compact evidence masks crucial for tasks such as MLLM hallucination repair.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.