Reproduce “MET: Masked Encoding for Tabular Data” on OpenResearch

About this paper

Kushal Majmundar, Sachin Goyal, Praneeth Netrapalli, Prateek Jain

A new self-supervised learning method, MET (Masked Encoding for Tabular Data), adapts masked autoencoders to tabular datasets, achieving state-of-the-art performance in downstream classification tasks. It efficiently leverages unlabeled data, demonstrating robust representation learning through adversarial reconstruction and comparable accuracy with significantly less labeled data than fully supervised methods.

Key insights

• Masked autoencoder architectures can be highly effective for self-supervised learning on tabular data when properly adapted, outperforming existing contrastive-based methods.
• Preserving the distinct semantic information of individual tabular features by concatenating their embeddings is crucial for learning high-quality representations, rather than relying on pooling strategies.
• Incorporating adversarial reconstruction loss during pre-training significantly enhances the robustness and discriminative power of the learned representations for tabular data.

Summary by alphaXiv.