Cultural Alien Sampler: Open-ended art generation balancing originality and coherence

TL;DR

This work tackles open-ended art generation by reframing idea generation as a discrete space of concepts and introducing the Cultural Alien Sampler (CAS), which explicitly balances internal coherence with cultural novelty. CAS uses two fine-tuned GPT-2 models—Concept Coherence and Cultural Context—to rank concept combinations, favoring those that are highly coherent yet culturally rare, and integrates this into a four-stage open-ended art agent. Through human evaluations and quantitative analyses, CAS outperforms random and GPT-4o baselines on originality and harmony and achieves broader conceptual exploration than GPT-4o, demonstrating that targeted, culturally alien sampling can unlock creative potential in autonomous agents. The approach reduces reliance on massive general models while delivering competitive creative quality and greater control for human creators, with future work aimed at collaborative, adaptive extensions and broader cultural coverage.

Abstract

In open-ended domains like art, autonomous agents must generate ideas that are both original and internally coherent, yet current Large Language Models (LLMs) either default to familiar cultural patterns or sacrifice coherence when pushed toward novelty. We address this by introducing the Cultural Alien Sampler (CAS), a concept-selection method that explicitly separates compositional fit from cultural typicality. CAS uses two GPT-2 models fine-tuned on WikiArt concepts: a Concept Coherence Model that scores whether concepts plausibly co-occur within artworks, and a Cultural Context Model that estimates how typical those combinations are within individual artists' bodies of work. CAS targets combinations that are high in coherence and low in typicality, yielding ideas that maintain internal consistency while deviating from learned conventions and embedded cultural context. In a human evaluation (N = 100), our approach outperforms random selection and GPT-4o baselines and achieves performance comparable to human art students in both perceived originality and harmony. Additionally, a quantitative study shows that our method produces more diverse outputs and explores a broader conceptual space than its GPT-4o counterpart, demonstrating that artificial cultural alienness can unlock creative potential in autonomous agents.

Figures (24)

• Figure 1: Evolution of generated images across generations 2, 5, and 10 using different sampling methods in the Inspiration Module. Concepts listed below each image were selected by the Prompt Compositor; purple concepts indicate newly introduced elements.
• Figure 2: Our Open-ended Art Agent (top) iterative pipeline consist of 4 main modules. In each iteration, the Inspiration Module takes the initial concept pool $\mathcal{P}_0$ and proposes a new concept to be added. The updated pool is then passed to the Prompt Compositor to create a full length text prompt which is passed, in turn, to the Image Generator to create the output image. The image and the prompt are evaluated using the novelty score and concepts that repeatedly fail to generate novel outputs are removed from the pool. Cultural Alien Sampler (bottom) is our proposed method for the inspiration Module. It comprising two fine-tuned transformer models that rank a list of proposed concepts to select ones that exhibit high coherence ranking but low cultural contextual ranking.
• Figure 3: Human evaluation results using Bradley–Terry skill parameters for originality and harmony. Significance levels: *, <0.05; **, <0.01; ***, <0.001; ****, <0.0001.
• Figure 4: Concept repetition across runs. Significance levels shown for key comparisons.
• Figure 5: Long-range simulation.A Example trajectories of each sampler (CAS, GPT, Free GPT, Random) for the starting concept pool Anime, Rabbit, projected into 2D t-SNE space with larger points and thicker edges indicating later generations. B Samplers differ in convergence behavior, measured by the maximum radius of explored image prompts (embedded with all-mpnet-base-v2) and convergence speed. Points denote individual runs; ellipses show distributions.