From Knots to Knobs: Towards Steerable Collaborative Filtering Using Sparse Autoencoders

TL;DR

This paper tackles the challenge of steering recommender systems while maintaining interpretability by embedding a sparse autoencoder (SAE) between the encoder and decoder of a collaborative filtering autoencoder (CFAE). The authors demonstrate that the nested CFAE+SAE yields sparse, monosemantic neuron activations that can be mapped to semantic concepts via item metadata and TF–IDF analysis, creating a controllable “knob” panel for steering recommendations through targeted neuron boosting. They compare Basic and TopK SAE variants, showing TopK SAEs preserve downstream accuracy with strong sparsity, and that concept–neuron mappings enable meaningful, in-process steering, albeit with architecture-dependent robustness. The work provides a modular, reproducible pipeline and highlights practical implications for user/editor control and model transparency in large-scale recommender systems, with potential extension to other embedding frontends and richer metadata.

Abstract

Sparse autoencoders (SAEs) have recently emerged as pivotal tools for introspection into large language models. SAEs can uncover high-quality, interpretable features at different levels of granularity and enable targeted steering of the generation process by selectively activating specific neurons in their latent activations. Our paper is the first to apply this approach to collaborative filtering, aiming to extract similarly interpretable features from representations learned purely from interaction signals. In particular, we focus on a widely adopted class of collaborative autoencoders (CFAEs) and augment them by inserting an SAE between their encoder and decoder networks. We demonstrate that such representation is largely monosemantic and propose suitable mapping functions between semantic concepts and individual neurons. We also evaluate a simple yet effective method that utilizes this representation to steer the recommendations in a desired direction.

Figures (5)

• Figure 1: Effects of SAE reconstruction. Results show SAEs trained on 1024-dimensional CFAE embeddings with various sparsity-inducing hyperparameters ($k, \ \lambda_1$). (1) TopK SAE (orange) is simple to use and achieves a superior sparsity--accuracy trade-off compared to Basic SAE (blue), which is very sensitive to hyperparameter selection. (2) TopK SAEs reconstruct cosine-like embeddings (ELSA) accuratelyand with minimal performance degradation. Variational embeddings (MultVAE) are difficult to reconstruct without sacrificing performance. Ablation: Replacing the $\mathrm{L}_2$ reconstruction loss in TopK SAE with cosine similarity loss (green) further improves the sparsity-accuracy trade-off for cosine-like embeddings (ELSA). However, this change no longer guarantees a small Euclidean distance between variational embeddings and their reconstructions, breaking the downstream performance in the case of MultVAE.
• Figure 2: KL divergence and entropy decrease of tags and neurons. For "tags to neurons" direction, relative entropy decrease is defined as $(H-H_t)/H$, where $H_t$ is the entropy of tag's distribution of neuron activations, while $H$ is the entropy of average distribution over all tags. The other direction is defined analogically.
• Figure 3: Effects of SAE-based steering on downstream recommendations for a particular user.
• Figure 4: Results of the steering procedure for varying intensities $\alpha$ and two concept-neuron mappings.
• Figure 5: Effects of SAE-based steering on user representations. As adjustment strength increases (indicated by saturation), user embeddings shift toward regions associated with representative items of the boosted concept.