ASAP: Interpretable Analysis and Summarization of AI-generated Image Patterns at Scale

TL;DR

ASAP addresses the need for interpretable, scalable analysis of AI-generated image patterns in the face of deceptive visuals. By distilling CLIP-based embeddings with a forget-to-spell projection and training a lightweight distiller, it creates compact representations that reveal pixel-level contributors to authenticity through gradient-based attributions. A coordinated visualization interface with Representation Overview, Image View, Pattern View, and Dimension View enables automatic pattern discovery, counterfactual analysis, and cross-image pattern summarization, demonstrated on GAN- and diffusion-model generated data. The system offers a generalizable HITL workflow that supports detection, explanation, and exploration of fake patterns, with potential to guide model development, prompt engineering, and broader applications beyond images.

Abstract

Generative image models have emerged as a promising technology to produce realistic images. Despite potential benefits, concerns grow about its misuse, particularly in generating deceptive images that could raise significant ethical, legal, and societal issues. Consequently, there is growing demand to empower users to effectively discern and comprehend patterns of AI-generated images. To this end, we developed ASAP, an interactive visualization system that automatically extracts distinct patterns of AI-generated images and allows users to interactively explore them via various views. To uncover fake patterns, ASAP introduces a novel image encoder, adapted from CLIP, which transforms images into compact "distilled" representations, enriched with information for differentiating authentic and fake images. These representations generate gradients that propagate back to the attention maps of CLIP's transformer block. This process quantifies the relative importance of each pixel to image authenticity or fakeness, exposing key deceptive patterns. ASAP enables the at scale interactive analysis of these patterns through multiple, coordinated visualizations. This includes a representation overview with innovative cell glyphs to aid in the exploration and qualitative evaluation of fake patterns across a vast array of images, as well as a pattern view that displays authenticity-indicating patterns in images and quantifies their impact. ASAP supports the analysis of cutting-edge generative models with the latest architectures, including GAN-based models like proGAN and diffusion models like the latent diffusion model. We demonstrate ASAP's usefulness through two usage scenarios using multiple fake image detection benchmark datasets, revealing its ability to identify and understand hidden patterns in AI-generated images, especially in detecting fake human faces produced by diffusion-based techniques.

Figures (5)

• Figure 1: We employ a two-step approach for detecting AI-generated images and identifying their underlying patterns. Step L (Learning) begins with encoding images using CLIP's image encoder (L1) and then proceeds to strip textual information via a 'forget-to-spell' projection (L2). Subsequently, we train a classifier that includes a distiller layer, which maps the 'forget-to-spell' representation to a distilled space (L3), and a classification head that predicts the authenticity of images (real or fake) based on this distilled representation (L4). In Step I (Identification), we focus on identifying influential pixel groups by computing token relevance using a gradient-based technique (I1), translating token maps to individual pixel maps (I2), and computing uniform contribution metrics for pixel groups based on the classifier's weights (I3).
• Figure 2: Our system introduces a novel cell glyph (G) to aggregate and visualize information for images within a cell, facilitating navigation in the representation space. The user can easily identify fakeness severity levels by hovering on the cell (G1). In this instance, the user examines the "brown/black horse head" cell with severe sensitivity (see Fig.\ref{['fig:violinplot']}).
• Figure 3: The user conducts a comparison between the dimension contributions (H2 and H5) of two "brown/black horse head" cells, specifically one with severe sensitivity (H1) and another with minor sensitivity (H4). The analysis uncovered a notable difference in dimension 2 across these cells. Further investigation into the pixel groups associated with dimension 2 revealed a strong correlation with pixels adjacent to or resembling horse eyes (noted in H3 and H6). It was observed that images in the severe sensitivity cell exhibit high values in dimension 2, whereas those in the minor sensitivity cell show low values. This pattern indicates that the realistic portrayal of horse eyes might significantly influence the detection between real and AI-generated horse head images.
• Figure 4: Leveraging Asap, the user identifies a cell containing misclassified images; upon examination, these are predominantly smiling faces. Through the concept view, it becomes evident that the mouth significantly influences the classification. However, a closer look at these misclassified images reveals that their smiles, though influencing classification were accurately detected by the model as their contribution are positive. This suggests the classifier was able to recognize 'fake smiles.' Further investigation shows that those images are misclassified due to high positive contribution from the forehead region
• Figure 5: The user identifies common patterns in misclassified real faces, including hair (P1) and background blur (P2), as well as factors contributing to the misclassification of fake images, such as "things on faces" (P3).