Reproduce “E2Former-V2: On-the-Fly Equivariant Attention with Linear Activation Memory” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lin Huang, Chengxiang Huang, Ziang Wang, Yiyue Du, Chu Wang, Haocheng Lu, Yunyang Li, Xiaoli Liu and 2 more
The authors present E2Former-V2, an equivariant molecular graph model that reduces attention memory by streaming neighbor contributions instead of materializing edge-level features. EAAS aligns geometric features to a local frame, while a fused Triton kernel computes attention and accumulates values on the GPU. In an ablation, throughput at 5,000 atoms reaches 4.24 samples/s; savings concern activations, not neighbor-dependent computation.
Key insights
- Equivariant convolution operators can be mathematically factorized into a node-centric form, theoretically reducing computational complexity from O(|E|) to O(|V|), which can be translated into practical, hardware-accelerated implementations.
- Leveraging pole sparsity by aligning coordinate systems to specific vectors can transform computationally intensive dense SO(3) tensor products into more efficient, sparse, permutation-like operations.
- Hardware-aware execution principles, similar to FlashAttention, can be successfully adapted for SO(3)-equivariant architectures to optimize on-chip SRAM usage and achieve linear activation memory scaling with respect to the number of nodes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.