Reproduce “Efficient Learning of Sparse Representations from Interactions” on OpenResearch

About this paper

Vojtěch Vančura, Martin Spišák, Rodrigo Alves, Ladislav Peška

Compressed ELSA learns high-dimensional sparse item embeddings directly from user-item interactions within a collaborative filtering autoencoder, achieving up to a 100x reduction in embedding memory footprint with minimal impact on recommendation accuracy. This approach also generates semantically coherent item segments from interaction data for enhanced interpretability.

Key insights

• Learning sparsity intrinsically during model training is more effective for developing compact recommender systems than applying post-hoc compression.
• Gradual pruning schedules that decrease sparsity after each epoch, combined with training restarts, yield higher quality sparse embeddings.
• High-dimensional sparse embeddings can achieve or exceed the recommendation accuracy of dense, uncompressed models while offering significant memory and computational efficiency gains.
• Sparse latent factors spontaneously capture semantically coherent item segments from interaction data, enabling interpretable and explainable recommendations without external metadata.

Summary by alphaXiv.