Clustering Internet Memes Through Template Matching and Multi-Dimensional Similarity

TL;DR

The paper tackles the problem of clustering Internet memes by proposing a template-based, multi-dimensional similarity framework that does not rely on predefined meme databases. It presents a four-step methodology: (1) extracting complementary global and local features across form, content, text, and identity; (2) constructing and aggregating feature-specific adjacency matrices to form a multimodal similarity graph; (3) identifying coherent templates via filtered clustering and graph-based modularity optimization; and (4) matching memes to templates with similarity vectors and incremental ranking. The approach demonstrates superior clustering consistency and coherence compared with baselines, and analyses show that combining multiple similarity dimensions aligns better with human judgments than any single dimension. Case studies illustrate the method’s ability to connect memes through multiple pathways (visual form, content, and identity), and experiments on KYM and Reddit data validate robustness and potential for dynamic meme retrieval and toxicity-aware analyses. The work advances meme understanding by integrating template-grounded structure with flexible, dimension-specific similarity, enabling more accurate, scalable, and adaptable meme clustering for moderation and sociocultural research. $The$ $method$ $relies$ $on$ $multi$-dimensional $adjacency$ $matrices$ and $Louvain$ $clustering$ to discover templates, while $A[i,j]$ captures pairwise similarities via aggregated feature sets and $s(d) = 1 - \tanh(d)$ maps distances to similarities, facilitating scalable analysis of evolving memes. The public release of the code further supports reproducibility and downstream applications such as similarity-based meme search and toxicity detection.

Abstract

Meme clustering is critical for toxicity detection, virality modeling, and typing, but it has received little attention in previous research. Clustering similar Internet memes is challenging due to their multimodality, cultural context, and adaptability. Existing approaches rely on databases, overlook semantics, and struggle to handle diverse dimensions of similarity. This paper introduces a novel method that uses template-based matching with multi-dimensional similarity features, thus eliminating the need for predefined databases and supporting adaptive matching. Memes are clustered using local and global features across similarity categories such as form, visual content, text, and identity. Our combined approach outperforms existing clustering methods, producing more consistent and coherent clusters, while similarity-based feature sets enable adaptability and align with human intuition. We make all supporting code publicly available to support subsequent research.

Figures (12)

• Figure 1: The original "Stonks" meme template, accompanied by example memes related through various dimensions: form, visual and textual content, and identity.
• Figure 2: Our methodology: feature extraction, adjacency matrix construction, template identification, and template matching.
• Figure 3: Mean weighted accuracy in the Imposter-Host task for various clustering methods across different numbers of total images clustered. Error bars indicate standard errors.
• Figure 4: Moving average accuracy of image matching using various feature sets. At each increment of additional images matched, the average is estimated over all clusters with that many images matched or fewer, using a rolling window. Highlighted areas represent the standard error of the mean.
• Figure 5: The percentage of clusters, deemed accurate by humans, for which various similarity dimensions were selected in response to: 'Select all the ways in which the memes [in this cluster] are related.'