Reproduce “A Parametric Linear Conic Projection for Structured Sparse Attention” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Michel Barlaud, Clement Genninasca, Maxime Zanella, Benoit Macq, Benoit Legat

The authors introduce PLCP, adaptive projection for sparse attention in transformer classifiers. It uses the scale-invariant Cone Alignment Index to select surviving entries and applies a closed-form projection. On QNLI at 90% attention sparsity, PLCP reaches 82.95% accuracy versus 74.05% for GSP and 73.49% for structured pruning. These classification results do not demonstrate end-to-end speedup.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.