Reproduce “Rationally Linearized MLPs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Arijit Sehanobish, Byeongchan Kim, Avinava Dubey, Min-hwan Oh, Krzysztof Choromanski
RaLiMP replaces costly MLP activation calculations with compact randomized inner products based on low-degree rational approximations, separating input-side and weight-side representations before combining them. In PTv3 semantic segmentation, replacing ten modules yields 77.54 mIoU on ScanNet, near the dense baseline’s 77.57. Measured MLP latency falls from 12.31 to 6.52 milliseconds; guarantees cover bounded pre-activations, with scale-dependent random-feature variance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.