Visual Polarization Measurement Using Counterfactual Image Generation
Mohammad Mosaffa, Omid Rafieian, Hema Yoganarasimhan
TL;DR
This paper develops PMCIG, a counterfactual-image framework to quantify visual political polarization in news media. It overcomes the information loss of traditional feature-extraction methods by generating two counterfactual image versions that differ only in a focal feature (eg, a smile) and tying polarization to a multi-modal news outlet prediction model, enabling estimation of the polarization parameter $\rho^T(p,y_1,y_2)$. The authors introduce Counterfactual Image Generation operators $\pi^1$ and $\pi^0$, a log-odds identification approach, and a cross-fitted algorithm that delivers high-granularity measures such as Conservative Visual Slant (CVS) and Overall Visual Polarization (OVP). Empirically, the framework reveals substantial visual slant and polarization across outlets and politicians, with strong validation against external slant indicators and meaningful associations with political geography. The method advances image-based bias measurement and provides a general, extensible tool for analyzing visual content in political discourse and beyond.
Abstract
Political polarization is a significant issue in American politics, influencing public discourse, policy, and consumer behavior. While studies on polarization in news media have extensively focused on verbal content, non-verbal elements, particularly visual content, have received less attention due to the complexity and high dimensionality of image data. Traditional descriptive approaches often rely on feature extraction from images, leading to biased polarization estimates due to information loss. In this paper, we introduce the Polarization Measurement using Counterfactual Image Generation (PMCIG) method, which combines economic theory with generative models and multi-modal deep learning to fully utilize the richness of image data and provide a theoretically grounded measure of polarization in visual content. Applying this framework to a decade-long dataset featuring 30 prominent politicians across 20 major news outlets, we identify significant polarization in visual content, with notable variations across outlets and politicians. At the news outlet level, we observe significant heterogeneity in visual slant. Outlets such as Daily Mail, Fox News, and Newsmax tend to favor Republican politicians in their visual content, while The Washington Post, USA Today, and The New York Times exhibit a slant in favor of Democratic politicians. At the politician level, our results reveal substantial variation in polarized coverage, with Donald Trump and Barack Obama among the most polarizing figures, while Joe Manchin and Susan Collins are among the least. Finally, we conduct a series of validation tests demonstrating the consistency of our proposed measures with external measures of media slant that rely on non-image-based sources.