Reproduce “Algebraic Machine Learning” on OpenResearch

About this paper

Fernando Martin-Maroto, Gonzalo G. de Polavieja

Fernando Martin-Maroto and Gonzalo G. de Polavieja introduce a parameter-free, fully discrete Algebraic Machine Learning framework that employs cardinal minimization instead of error function minimization. This method generates interpretable concept definitions, demonstrates generalization without overfitting on MNIST, and integrates formal rules for unsupervised learning tasks like the N-Queens problem.

Key insights

• Generalization can be achieved inherently and without overfitting by finding the algebraically minimal representation (cardinal minimization) that satisfies the training data, eliminating the need for explicit regularization.
• There is a formal and empirically validated inverse relationship between a model's compression ratio (ratio of training examples to atoms) and its expected test error, directly linking model simplicity to generalization capability.
• Learning based on discrete algebraic structures naturally leads to the formation of human-interpretable concepts and formal definitions directly from data, offering transparency not present in most statistical models.

Summary by alphaXiv.