Reproduce “SO(3)-RoPE for Spherical Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Christian Libner, Chase van de Geijn, Alexander S. Ecker, Maurice Weiler
SO3ViT predicts next-timestep fluid states on a rotating sphere using attention that encodes relative rotations between spherical patches. It lifts patches to SO(3), transforms query and key features by patch orientation, and uses FlashAttention. Across tested configurations, it has a better ℓ2-error versus median-epoch-time Pareto frontier than S2Transformer; this comparison does not isolate the positional encoding or establish broader performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.