Reproduce “Foveation-Guided Dynamic Token Selection for Robust and Efficient Vision Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ibrahim Batuhan Akkaya, Kishaan Jeeveswaran, Bahram Zonooz, Elahe Arani
Researchers from NavInfo Europe and Eindhoven University of Technology developed the Foveated Dynamic Transformer (FDT), a Vision Transformer architecture that emulates human foveated sampling to dynamically select and process informative tokens. This approach improves robustness against adversarial attacks, natural corruptions, and shortcut learning, while reducing computational operations by over 34% compared to baseline models.
Key insights
- Biologically inspired foveation and dynamic token selection can inherently improve a model's robustness and computational efficiency.
- Adaptive, input-dependent processing allows for targeted allocation of computational resources, focusing on salient image regions.
- Robustness against diverse perturbations can emerge from architectural design inspired by biological vision, rather than solely from explicit adversarial or corruption training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.