Reproduce “Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hao Li, Jingkuan Song, Lianli Gao, Xiaosu Zhu, Heng Tao Shen
A Prototype-based Aleatoric Uncertainty Quantification (PAU) framework is introduced to model and quantify data-inherent ambiguity in cross-modal retrieval. This method, leveraging Dempster-Shafer Theory and Subjective Logic, enhances retrieval trustworthiness and achieves performance gains of up to 8.8% R@1 on video-to-text benchmarks by effectively handling uncertain data.
Key insights
- Inherent data ambiguity (aleatoric uncertainty) in cross-modal retrieval can be precisely quantified by measuring an instance's similarity to a diverse set of semantic prototypes using evidential deep learning.
- Employing Dempster-Shafer Theory and Subjective Logic provides a more robust measure of uncertainty compared to information entropy, distinguishing between genuine ambiguity and mere weak matches.
- Actively learning and incorporating uncertainty into the retrieval process, especially through a re-ranking mechanism, improves the trustworthiness and overall accuracy of predictions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.