One Does Not Simply Meme Alone: Evaluating Co-Creativity Between LLMs and Humans in the Generation of Humor

TL;DR

The paper investigates how LLMs function as co-creators in humor-rich meme generation via a three-arm user study (human-only, human-AI collaboration, AI-only). It finds that LLM assistance increases idea production and lowers perceived effort but does not consistently improve meme quality, while AI-only memes perform best on average; top-performing memes often arise from human input, especially in humor. The authors discuss the tension between productivity and ownership in human-AI collaboration and argue for better-structured, collaborative AI interfaces to preserve human touch. Overall, the work highlights both the potential and limits of LLMs in creative humor tasks and calls for careful design of AI tools to complement human creativity rather than replace it.

Abstract

Collaboration has been shown to enhance creativity, leading to more innovative and effective outcomes. While previous research has explored the abilities of Large Language Models (LLMs) to serve as co-creative partners in tasks like writing poetry or creating narratives, the collaborative potential of LLMs in humor-rich and culturally nuanced domains remains an open question. To address this gap, we conducted a user study to explore the potential of LLMs in co-creating memes - a humor-driven and culturally specific form of creative expression. We conducted a user study with three groups of 50 participants each: a human-only group creating memes without AI assistance, a human-AI collaboration group interacting with a state-of-the-art LLM model, and an AI-only group where the LLM autonomously generated memes. We assessed the quality of the generated memes through crowdsourcing, with each meme rated on creativity, humor, and shareability. Our results showed that LLM assistance increased the number of ideas generated and reduced the effort participants felt. However, it did not improve the quality of the memes when humans collaborated with LLM. Interestingly, memes created entirely by AI performed better than both human-only and human-AI collaborative memes in all areas on average. However, when looking at the top-performing memes, human-created ones were better in humor, while human-AI collaborations stood out in creativity and shareability. These findings highlight the complexities of human-AI collaboration in creative tasks. While AI can boost productivity and create content that appeals to a broad audience, human creativity remains crucial for content that connects on a deeper level.

Figures (7)

• Figure 1: Mapping of Meme Templates to Topics (Work, Food, Sports) in the Study.
• Figure 2: User Interface Overview:Baseline Ideation, Ideation with Chat Interface, Favorite Selection, and Final Image Creation.
• Figure 3: Meme Generation Workflow: Human (Baseline), Human-AI Collaboration, and AI-Driven Creation.
• Figure 4: Meme Evaluation Workflow: This diagram illustrates the evaluation process of memes created by humans, human-AI collaboration, and AI-driven approaches.
• Figure 5: Participants using the LLM were able to produce significantly more ideas than participants who had no external support, according to the Mann-Whitney-U test (***: $p < 0.001$)