Reproduce “Probabilistic Invariant Learning with Randomized Linear Classifiers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Leonardo Cotta, Gal Yehuda, Assaf Schuster, Chris J. Maddison
Researchers introduce Randomized Linear Classifiers (RLCs), a new model class that leverages probabilistic universal approximation and invariance to address the resource limitations of deterministic invariant neural networks. This approach proves to be parameter-efficient and robust, outperforming Deep Sets and GNNs on tasks like set sorting and graph connectivity while maintaining parameter counts independent of input size.
Key insights
- Randomness can serve as a computational resource, enabling probabilistic universal approximation and invariance to circumvent the resource tradeoffs inherent in deterministic models.
- G-invariant RLCs are achieved by ensuring the *distribution* of generated classifier weights is G-invariant, rather than requiring the generating neural network itself to be invariant.
- The parameter count for invariant RLCs can be independent of the input size (e.g., set size, graph size), offering significant parameter efficiency over deterministic counterparts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.