Reproduce “Random Forest Kernel for High-Dimension Low Sample Size Classification” on OpenResearch

About this paper

Lucca Portes Cavalheiro, Simon Bernard, Jean Paul Barddal, Laurent Heutte

The Random Forest SVM (RFSVM) method integrates Random Forest-derived similarity as a positive semi-definite kernel for Support Vector Machines, enhancing classification accuracy and robustness for High Dimension, Low Sample Size (HDLSS) problems. It consistently outperformed several state-of-the-art HDLSS and general-purpose classifiers across 40 diverse datasets, particularly in very-HDLSS scenarios.

Key insights

• Learning based on similarities between instances is a robust approach for small-sized datasets, capable of implicitly capturing complex relationships without direct reliance on high-dimensional feature spaces for distance computations.
• The Random Forest similarity measure is inherently robust to high dimensions and reflects class belonging, making it an effective and unbiased kernel for SVMs in HDLSS contexts, unlike traditional distance metrics or standard kernels.
• Framing HDLSS classification through an HDLSS-robust similarity kernel provides a 'genuine' solution, offering a strong alternative to dimensionality reduction and mitigating issues like SVM's data-piling phenomenon.

Summary by alphaXiv.