Reproduce “Concrete Autoencoders for Differentiable Feature Selection and Reconstruction” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Abubakar Abid, Muhammad Fatih Balin, James Zou

Concrete Autoencoders introduce an end-to-end differentiable method for unsupervised discrete feature selection, enabling the identification of a minimal set of original features that can accurately reconstruct high-dimensional data. This approach achieved lower reconstruction errors compared to established methods and demonstrated a 20% reduction in features required for genomic imputation.

Key insights

  • The Concrete distribution provides a differentiable relaxation of discrete choices, allowing gradient-based optimization directly through the feature selection process.
  • A carefully designed temperature annealing schedule is crucial for balancing the exploration of feature combinations with convergence to a stable, discrete set of selected features.
  • Optimizing for reconstruction error in an unsupervised setting can effectively identify features that are also highly informative for various downstream supervised tasks without needing explicit labels.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.