Reproduce “Rethinking Vision Architectures with Gated Linear Attention and KAN” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ali Mehizel, Oussama Khaldi
LKAT pairs gated linear attention (GLA) for mixing image tokens with a two-layer Kolmogorov–Arnold Network (KAN) for channel mixing, using fused radial-basis-function (RBF) computation to avoid a large GPU-memory intermediate. On ImageNet-100, LKAT-B reaches 80.14% Top-1 accuracy versus 78.72% for ViT-B/16 after shared 300-epoch training. This base-model result does not isolate either component’s contribution or establish end-to-end speed.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.