Prompt injections as a tool for preserving identity in GAI image descriptions

TL;DR

This work addresses the harms indirect users endure from GAI-generated image descriptions, where biases and misrepresentation can occur without direct interaction. It investigates prompt injections as a bottom-up, user-empowering technique to preserve identity cues within GAI outputs, demonstrated through a case study involving a non-binary, autistic individual and three image-description tools. The findings indicate that carefully crafted prompt injections can surface and persist identity markers, though effectiveness varies by tool and injection type, with some risks of stereotyping. Overall, the study highlights prompt injections as a practical, participatory method for mitigating misrepresentation and enabling indirect user agency in GAI interactions, while underscoring ethical considerations and the need for further refinement.

Abstract

Generative AI risks such as bias and lack of representation impact people who do not interact directly with GAI systems, but whose content does: indirect users. Several approaches to mitigating harms to indirect users have been described, but most require top down or external intervention. An emerging strategy, prompt injections, provides an empowering alternative: indirect users can mitigate harm against them, from within their own content. Our approach proposes prompt injections not as a malicious attack vector, but as a tool for content/image owner resistance. In this poster, we demonstrate one case study of prompt injections for empowering an indirect user, by retaining an image owner's gender and disabled identity when an image is described by GAI.