The Persistence of Cultural Memory: Investigating Multimodal Iconicity in Diffusion Models

TL;DR

This work tackles the ambiguity between generalization and memorization in text-to-image diffusion models by introducing multimodal iconicity, a framework that separates recognition of cultural references from their visual realization. The authors propose a prompt-agnostic evaluation pipeline using CRA, CRC, VR, and CRT, enabling nuanced analysis of how models reproduce or transform culturally shared image-text associations. Across five diffusion models and 767 Wikidata-derived references, they show that multimodal iconicity correlates with textual uniqueness, reference popularity, and creation date, and that models balance recognition and transformation in diverse ways. The study advances evaluation beyond simple similarity metrics, highlighting the role of cultural context in model behavior and informing approaches to copyright, privacy, and content reinterpretation in generative AI.

Abstract

Our work addresses the ambiguity between generalization and memorization in text-to-image diffusion models, focusing on a specific case we term multimodal iconicity. This refers to instances where images and texts evoke culturally shared associations, such as when a title recalls a familiar artwork or film scene. While prior research on memorization and unlearning emphasizes forgetting, we examine what is remembered and how, focusing on the balance between recognizing cultural references and reproducing them. We introduce an evaluation framework that separates recognition, whether a model identifies a reference, from realization, how it depicts it through replication or reinterpretation, quantified through measures capturing both dimensions. By evaluating five diffusion models across 767 Wikidata-derived cultural references spanning static and dynamic imagery, we show that our framework distinguishes replication from transformation more effectively than existing similarity-based methods. To assess linguistic sensitivity, we conduct prompt perturbation experiments using synonym substitutions and literal image descriptions, finding that models often reproduce iconic visual structures even when textual cues are altered. Finally, our analysis shows that cultural alignment correlates not only with training data frequency, but also textual uniqueness, reference popularity, and creation date. Our work reveals that the value of diffusion models lies not only in what they reproduce but in how they transform and recontextualize cultural knowledge, advancing evaluation beyond simple text-image matching toward richer contextual understanding.

Figures (27)

• Figure 1: Example generations from Stable Diffusion XL for culturally iconic references like The Persistence of Memory, Lady with an Ermine, Atom Heart Mother, The Unforgettable Fire, The Godfather, A Clockwork Orange, and 12 Angry Men.
• Figure 2: Recognition vs. Transformation in Multimodal Iconicity. Generations from three diffusion models illustrate how they respond to the prompt “The Dark Side of the Moon.” The vertical axis (Recognition) indicates whether the model evokes the intended cultural reference, while the horizontal axis (Transformation) reflects the degree of visual reinterpretation.
• Figure 3: Framework for evaluating multimodal iconicity. Cultural reference prompts generate images evaluated along two dimensions: Recognition (CRA), measuring alignment with reference images via CLIP, and Realization (VI), measuring how independently the model recreates them using DINOv3 patch analysis. The resulting Cultural Reference Transformation (CRT) metric captures both a model’s ability to identify cultural references and the manner in which it visually realizes them.
• Figure 4: CRA--VR relationship across diffusion models for static (top) and dynamic (bottom) cultural references. Each point corresponds to a single reference, showing the model's recognition ability (CRA) and degree of visual reuse (VR). Darker points indicate high CRT ($> 0.8$), where the model recognizes a reference while generating an independent realization. Percentages below each subplot denote the proportion of references with high CRT among all aligned ones.
• Figure 5: Example of images generated from the prompt Physical Graffiti using three diffusion models, SDXL, SD3, and Flux Schnell, shown alongside the iconic cultural reference image.