Reproduce “Knowing When Not to Answer: Evaluating Abstention in Multimodal Reasoning Systems” on OpenResearch

About this paper

Nishanth Madhusudhan, Vikas Yadav, Alexandre Lacoste

ServiceNow Research introduced MM-AQA, a new benchmark and evaluation framework, to assess multimodal reasoning systems' ability to abstain from answering when evidence is insufficient. The study found that frontier Vision-Language Models and Multi-Agent Systems rarely abstain under standard prompting and exhibit an accuracy-abstention trade-off, revealing a significant performance gap to human oracle performance (MCC A 0.83).

Key insights

• Current frontier VLMs and MAS exhibit poor calibration, rarely abstaining under standard prompting and frequently hallucinating when faced with unanswerable queries.
• Models possess latent uncertainty signals that are suppressed by default decoding, as explicit 'extreme abstain' clauses significantly improve Unanswerable Accuracy (UAC) with only marginal accuracy degradation.
• Multi-agent systems improve abstention primarily through a Verifier agent, but this introduces an accuracy-abstention trade-off, with no configuration simultaneously exceeding 65% on both Answer Accuracy (AAC) and UAC.

Summary by alphaXiv.