Recommender system in X inadvertently profiles ideological positions of users

TL;DR

The results show that the platform's recommender system produces a spatial ordering of users that is highly correlated with their Left-Right positions, and that cannot be explained by socio-demographic attributes, which opens new possibilities for studying the interaction between human and AI systems.

Abstract

Studies on recommendations in social media have mainly analyzed the quality of recommended items (e.g., their diversity or biases) and the impact of recommendation policies (e.g., in comparison with purely chronological policies). We use a data donation program, collecting more than 2.5 million friend recommendations made to 682 volunteers on X over a year, to study instead how real-world recommenders learn, represent and process political and social attributes of users inside the so-called black boxes of AI systems. Using publicly available knowledge on the architecture of the recommender, we inferred the positions of recommended users in its embedding space. Leveraging ideology scaling calibrated with political survey data, we analyzed the political position of users in our study (N=26,509 among volunteers and recommended contacts) among several attributes, including age and gender. Our results show that the platform's recommender system produces a spatial ordering of users that is highly correlated with their Left-Right positions (Pearson rho=0.887, p-value < 0.0001), and that cannot be explained by socio-demographic attributes. These results open new possibilities for studying the interaction between human and AI systems. They also raise important questions linked to the legal definition of algorithmic profiling in data privacy regulation by blurring the line between active and passive profiling. We explore new constrained recommendation methods enabled by our results, limiting the political information in the recommender as a potential tool for privacy compliance capable of preserving recommendation relevance.

Figures (4)

• Figure 1: Schematic representation of the population of X users for the study and the inference of their position in the embedding space of X's recommender system. (A) Population of the study (data volunteers $\mathcal{V}$ and recommended users $\mathcal{U}$) subtending networks for friend recommendation (in orange arrows) and following relations (in purple arrows), collected respectively with a data donation program using a browser plug-in and X's API. (B) Schematic representation of the embedding space with every user $i$ in the population $\mathcal{E}$ having a position $\phi_i$ in the 256-dimensional embedding.
• Figure 2: Characterization of the population in the study. (A) Distribution of Left-Right ideological leaning of X users segmented by age and gender. (B) Pairwise Spearman correlations between the inferred attributes of users (* indicates p-value $<0.01$, ** indicates p-value $<0.001$). The correlation between Left-Right leaning and degree of interest in news is -0.14**. The density plots for the degree of interest in news of users in different age groups illustrates a comparatively higher correlation between this attribute and age.
• Figure 3: (A) Three-dimensional representation of directions in the 256-dimensional embedding space of the recommender system that maximize the correlation with attributes of users, identified using Canonical Correlation Analysis (CCA). The three-dimensional visualization was obtained through a Locally Preserving Projection LPP. Vector norms of directions associated with attributes correspond to absolute correlations with these attributes, reported in the adjacent table (all p-values $< 0.0001$). (B) Absolute cosine similarity computed in the 256-dimensional embedding space between directions in the embedding space identified with CCA and that maximize correlations with attributes of users (* indicates p-value $<0.01$). (C) Comparison of attributes of users along the CCA directions in the embedding that maximize correlations with three features: Left-Right leaning $\mathbf{w}_{LR}$, age $\mathbf{w}_{age}$, and gender $\mathbf{w}_{gender}$.
• Figure 4: Constraining user recommendations from leveraging left-right political information. (A) Diagram of the intervention procedure. After identifying the direction in the latent space that maximally correlates with users' left-right political leaning, we iteratively project user representations onto the orthogonal subspace. (B) Effects on recommendations of constraining political information in the embedding of the recommender systems. For each user in the population, we compute 50 friend recommendations before (in blue) and after (in green) the intervention and compute the impact the distribution of the ideological diversity of recommendations as the standard deviation on the Left-Right scale, and the relevance as the similarity in interest in news between recipient of recommendations and recommended friends.