Creative Problem Solving in Large Language and Vision Models -- What Would it Take?

TL;DR

Preliminary experiments are presented showing how CC principles can be applied to address a key limitation of large language and vision models, i.e., creative problem solving.

Abstract

We advocate for a strong integration of Computational Creativity (CC) with research in large language and vision models (LLVMs) to address a key limitation of these models, i.e., creative problem solving. We present preliminary experiments showing how CC principles can be applied to address this limitation. Our goal is to foster discussions on creative problem solving in LLVMs and CC at prestigious ML venues. Our code is available at: https://github.com/lnairGT/creative-problem-solving-LLMs

Figures (7)

• Figure 1: Computational Creativity can help address a gap in the intelligence of present-day LLVMs, elevating their ingenuity through creative problem solving.
• Figure 2: Object replacement group: Average accuracies and standard deviations of the models across ten different sets of randomly chosen objects.
• Figure 3: Complete test set of objects used in the experiments.
• Figure 4: Object replacement test: Using the same prompts as for the nominal group. Random selection of a replacement object achieves $\approx$30% overall accuracy.
• Figure 5: Object replacement test: Accuracies when the prompts are augmented with object affordance information.