Reproduce “Revisiting Score Function Estimators for $k$-Subset Sampling” on OpenResearch

About this paper

Klas Wijk, Ricardo Vinuesa, Hossein Azizpour

Revisiting score function estimators for k-subset sampling, this research develops a method that provides exact discrete samples and unbiased gradient estimates for feature selection. The proposed SFESS-V approach, incorporating variance reduction, consistently achieves superior performance and faster training times compared to existing relaxed sampling and pathwise gradient techniques across image reconstruction and classification tasks.

Key insights

• Score function estimators can be effectively adapted for k-subset sampling, offering both exact discrete samples and theoretically unbiased gradient estimates.
• The Discrete Fourier Transform provides a computationally efficient way to handle the Poisson binomial distribution, which is a key component of the k-subset score function.
• Variance reduction techniques, such as control variates, are crucial for transforming vanilla score function estimators into practically viable and high-performing methods for k-subset sampling.

Summary by alphaXiv.