Reproduce “Uncertainty Estimation by Fisher Information-based Evidential Deep Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Danruo Deng, Guangyong Chen, Yang Yu, Furui Liu, Pheng-Ann Heng

Fisher Information-based Evidential Deep Learning (I-EDL) refines uncertainty quantification in deep learning by using Fisher Information to dynamically reweight loss terms, addressing over-penalization for uncertain samples in classical Evidential Deep Learning. The method achieves state-of-the-art performance in confidence evaluation, OOD detection, and few-shot learning, improving classification accuracy by up to 9.31% in few-shot settings.

Key insights

  • Fisher Information can quantify the "informativeness" of evidence for different classes, enabling dynamic adjustment of loss contributions: lower information (higher evidence) leads to greater leniency.
  • Dynamically reweighting loss terms via the inverse Fisher Information Matrix allows the network to maintain evidence for multiple plausible classes for ambiguous samples, overcoming the over-penalization issue of classical EDL.
  • The integration of a PAC-Bayesian bound and a determinant penalty regularizes the learning process, leading to improved generalization and preventing overconfidence by discouraging excessively sharp evidence for single classes.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.