Reproduce “Near-Optimal Glimpse Sequences for Improved Hard Attention Neural Network Training” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
William Harvey, Michael Teng, Frank Wood
The authors generate image-glimpse routes by estimating which patch is expected to leave least class uncertainty, then use some routes to supervise hard-attention classifiers. Across 40 CelebA-HQ attribute tasks, PS-NOGS averaged 3,500 iterations to reach validation cross-entropy within 0.01 of best, versus 24,000 for RAM, with mean test accuracy of 87.1% versus 86.7%. Routes are costly to generate; the result concerns iterations, not total computation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.