Reproduce “Flow-based Spectral Kernel Learning for Nonstationary Attention” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zi Yang, Ying Li, Alejandro Lancho, Darío C. Larese, Pablo M. Olmos, Michael Minyi Zhang
FSKA learns a nonstationary attention similarity rule with a normalizing flow that models a bivariate spectral density. Random features retain linear attention computation, while learned query–key projections are removed. Across four full-document classification datasets, FSKA reports the highest mean macro-F1, but Mamba has higher accuracy on SCOTUS. SCOTUS macro-F1 is 39.68 ± 0.71%; mean differences do not establish statistical significance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.