Reproduce “TabNSM: Neural Sparse Mixer for Tabular Regression” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ali Eslamian, Qiang Cheng

The TabNSM framework introduces a scalable neural sparse mixer for high-dimensional tabular regression, integrating an Adaptive Sparse Interaction Module (ASIM) with a multi-stage head, GridLoss, and RISE sampling to robustly learn from complex data. It achieved the lowest RMSE on seven out of nine diverse real-world benchmarks, demonstrating statistically significant performance gains over 38 baselines.

Key insights

  • Instance-adaptive sparse feature selection is crucial for robust high-dimensional tabular regression, demonstrating superiority over dense attention and efficient linear mixing.
  • The combination of ordinal-aware loss functions (GridLoss) and difficulty-aware sampling (RISE) enhances training stability and predictive accuracy for continuous targets.
  • Deep learning models for tabular data can achieve near-linear computational scaling with feature dimension while maintaining strong performance through specialized sparse interaction modules.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.