Reproduce “Raising the Ceiling: Better Empirical Fixation Densities for Saliency Benchmarking” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Susmit Agrawal, Jannis Hollman, Matthias Kümmerer
Researchers at the Tübingen AI Center developed a refined methodology for estimating empirical fixation densities, foundational for saliency benchmarking, which combines adaptive bandwidth Kernel Density Estimation with saliency model regularization. This approach yields substantially higher interobserver consistency estimates, increasing log-likelihood by up to 0.4 bit/fix, and provides a more robust "gold standard" for evaluating computational models of visual attention.
Key insights
- Interobserver consistency (IOC) is not a static ceiling but an evolving estimate dependent on the quality of the density estimation methodology, and improved methods can significantly increase its value.
- Adaptive bandwidth KDE and saliency model regularization are key drivers for improving fixation density estimates, with adaptive bandwidth capturing multi-scale spatial structure and saliency regularization providing valuable semantic priors, especially for sparse data.
- Commonly used pooled fixation densities significantly overfit to the data, artificially inflating performance scores by 0.30–0.35 bit/fix compared to properly cross-validated estimates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.