Alien Recombination: Exploring Concept Blends Beyond Human Cognitive Availability in Visual Art

TL;DR

The Alien Recombination method is presented, a novel approach utilizing fine-tuned large language models to identify and generate concept combinations that lie beyond human cognitive availability, suggesting that cognitive unavailability is a promising metric for optimizing artistic novelty, outperforming merely temperature scaling without additional evaluation criteria.

Abstract

While AI models have demonstrated remarkable capabilities in constrained domains like game strategy, their potential for genuine creativity in open-ended domains like art remains debated. We explore this question by examining how AI can transcend human cognitive limitations in visual art creation. Our research hypothesizes that visual art contains a vast unexplored space of conceptual combinations, constrained not by inherent incompatibility, but by cognitive limitations imposed by artists' cultural, temporal, geographical and social contexts. To test this hypothesis, we present the Alien Recombination method, a novel approach utilizing fine-tuned large language models to identify and generate concept combinations that lie beyond human cognitive availability. The system models and deliberately counteracts human availability bias, the tendency to rely on immediately accessible examples, to discover novel artistic combinations. This system not only produces combinations that have never been attempted before within our dataset but also identifies and generates combinations that are cognitively unavailable to all artists in the domain. Furthermore, we translate these combinations into visual representations, enabling the exploration of subjective perceptions of novelty. Our findings suggest that cognitive unavailability is a promising metric for optimizing artistic novelty, outperforming merely temperature scaling without additional evaluation criteria. This approach uses generative models to connect previously unconnected ideas, providing new insight into the potential of framing AI-driven creativity as a combinatorial problem.

Figures (11)

• Figure 1: Comparison of images generated using random sampling versus Alien sampling with the Art model for the input sequence "Insect". The Art model, fine-tuned on WikiArt concepts saleh2015large, combines likely concepts to produce artworks, with novelty controlled by temperature. Random sampling selects concept combinations arbitrarily, while Alien sampling ranks combinations based on cognitive unavailability and artistic fit, generating images from top-ranked sequences. For the full Alien Recombination method, see Figure \ref{['fig:architecture']}.
• Figure 2: Comparison on the novelty of the generated sequences with respect to the artworks and the cognitive availability for multiple values of $\beta$
• Figure 3: GPT-4 pairwise evaluation
• Figure 4: Schematic representation of the Alien Recombination method. The Art model generates N sequences from an input sequence. These sequences are ranked by both the Art model (ascending perplexity) and the Cognitive Availability model (descending perplexity). The ranking and selection process, termed "Alien sampling", employs a weighted rank aggregation method parameterized by $\beta$. Increasing $\beta$ prioritizes sequences that are more distant from what is cognitively available, thus enhancing alieness. The top-$k$ sequences (user-defined $k$) resulting from this fused ranking are then processed by a text-to-image model (DALL-E ramesh2022hierarchical in this study) to generate images, using the prompt: A painting that contains the concepts: <input sequence + generated sequence>.
• Figure 5: Examples of images generated with the Alien Recombination method for the input "Romanticism Mountain". The returned sequence by the Alien sampling is written on top of the respective image.