Reproduce “Attend to Anything: Foundation Model for Unified Human Attention Modeling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenzhuo Zhao, Ronghao Xian, Keren Fu, Qijun Zhao
The Attend to Anything Model (AAM) unifies human attention modeling across image, video, and audio-visual tasks by representing attention as a hierarchical cognitive entailment in hyperbolic space and temporal dynamics using Fokker–Planck equations. It consistently outperforms state-of-the-art methods across 16 benchmarks with an average performance gain of 6% and achieves a 4x video inference speedup.
Key insights
- Human attention variations can be effectively modeled as a hierarchical cognitive entailment process, allowing a single model to generalize across diverse scenes and contexts.
- Hyperbolic geometry provides a suitable framework for representing the hierarchical organization of attentional modulation, from general cognitive priors to specific visual instantiations.
- Temporal attention dynamics in videos can be accurately and efficiently captured by formulating them as a continuous diffusive process governed by the Fokker–Planck equation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.