Reproduce “From Knots to Knobs: Towards Steerable Collaborative Filtering Using Sparse Autoencoders” on OpenResearch

About this paper

Martin Spišák, Ladislav Peška, Petr Škoda, Vojtěch Vančura, Rodrigo Alves

Researchers integrated Sparse Autoencoders with Collaborative Filtering Autoencoders to convert dense user embeddings into sparse, interpretable "knobs," enabling fine-grained control over recommendation generation. This approach successfully disentangled monosemantic concepts from user interaction data, maintaining high recommendation accuracy (e.g., 94% nDCG@20 with 32 neurons for ELSA) and allowing targeted steering.

Key insights

• Sparse Autoencoders can organically learn interpretable, monosemantic features ("knobs") from purely interaction-based user embeddings generated by CFAEs, without relying on item metadata during SAE training.
• The efficacy of sparse reconstruction and steerability is significantly influenced by the Collaborative Filtering Autoencoder's architecture; models yielding cosine-like embeddings (e.g., ELSA) are more robust to sparsity than those with variational embeddings (e.g., MultVAE).
• Modifying individual sparse neurons enables plausible and effective fine-grained steering of recommendations, allowing for personalized adjustments towards specific semantic concepts.

Summary by alphaXiv.