Deep Learning for Climate Action: Computer Vision Analysis of Visual Narratives on X
Katharina Prasse, Marcel Kleinmann, Inken Adam, Kerstin Beckersjuergen, Andreas Edte, Jona Frroku, Timotheus Gumpp, Steffen Jung, Isaac Bravo, Stefanie Walter, Margret Keuper
TL;DR
The paper addresses the challenge of understanding climate-change discourse on social media when image content is increasingly central and access to data is restricted. It proposes a multi-modal pipeline that combines image classification, object detection, and sentiment analysis on a large 2019 dataset of 730k climate-related tweets with images, augmented by foundation models and an interactive GUI. Key findings include identifiable visual frames and general positivity in image sentiment, contrasted with more varied text sentiment and notable cross-modal divergence, as well as limitations in current sentiment models. The work contributes open-source code and a GUI to support reproducible, AI-enabled research at the intersection of climate communication, social media, and computer vision.
Abstract
Climate change is one of the most pressing challenges of the 21st century, sparking widespread discourse across social media platforms. Activists, policymakers, and researchers seek to understand public sentiment and narratives while access to social media data has become increasingly restricted in the post-API era. In this study, we analyze a dataset of climate change-related tweets from X (formerly Twitter) shared in 2019, containing 730k tweets along with the shared images. Our approach integrates statistical analysis, image classification, object detection, and sentiment analysis to explore visual narratives in climate discourse. Additionally, we introduce a graphical user interface (GUI) to facilitate interactive data exploration. Our findings reveal key themes in climate communication, highlight sentiment divergence between images and text, and underscore the strengths and limitations of foundation models in analyzing social media imagery. By releasing our code and tools, we aim to support future research on the intersection of climate change, social media, and computer vision.