Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation

TL;DR

This work tackles Leap-of-Thought (LoT) in large language models by introducing Oogiri-GO, a large, multimodal, multilingual humor dataset, and the Creative Leap-of-Thought (CLoT) framework. CLoT comprises associable instruction tuning and explorative self-refinement to foster non-sequential, associative creativity beyond traditional Chain-of-Thought methods. Empirical results show that CLoT improves LoT capabilities across multiple tasks and models, and transfers to other creative benchmarks like CGG and DAT, with positive human evaluations. The dataset, code, and trained models are released to promote broader exploration of LoT in creative AI applications.

Abstract

Chain-of-Thought (CoT) guides large language models (LLMs) to reason step-by-step, and can motivate their logical reasoning ability. While effective for logical tasks, CoT is not conducive to creative problem-solving which often requires out-of-box thoughts and is crucial for innovation advancements. In this paper, we explore the Leap-of-Thought (LoT) abilities within LLMs -- a non-sequential, creative paradigm involving strong associations and knowledge leaps. To this end, we study LLMs on the popular Oogiri game which needs participants to have good creativity and strong associative thinking for responding unexpectedly and humorously to the given image, text, or both, and thus is suitable for LoT study. Then to investigate LLMs' LoT ability in the Oogiri game, we first build a multimodal and multilingual Oogiri-GO dataset which contains over 130,000 samples from the Oogiri game, and observe the insufficient LoT ability or failures of most existing LLMs on the Oogiri game. Accordingly, we introduce a creative Leap-of-Thought (CLoT) paradigm to improve LLM's LoT ability. CLoT first formulates the Oogiri-GO dataset into LoT-oriented instruction tuning data to train pretrained LLM for achieving certain LoT humor generation and discrimination abilities. Then CLoT designs an explorative self-refinement that encourages the LLM to generate more creative LoT data via exploring parallels between seemingly unrelated concepts and selects high-quality data to train itself for self-refinement. CLoT not only excels in humor generation in the Oogiri game but also boosts creative abilities in various tasks like cloud guessing game and divergent association task. These findings advance our understanding and offer a pathway to improve LLMs' creative capacities for innovative applications across domains. The dataset, code, and models will be released online. https://zhongshsh.github.io/CLoT/.

Figures (29)

• Figure 1: Comparison between (multimodal) large language model (LLM, red) and its CLoT-integrated version ( blue) for Oogiri-style multimodal humor generation. According to the model input that can be image, text or both, there are three Oogiri tasks, "Image&Text to Text (IT2T)", "Image to Text (I2T)", and "Text to Text (T2T)”, where text can be English (EN), Chinese (CN), and Japanese (JP). "@" denotes translations in English. The baseline LLM is Qwen-VL Qwen-VL. While humor is subjective, these examples demonstrate CLoT's leap-of-thought capacity of using excellent creative thinking to produce high-quality humor responses. See more examples in Appendix.
• Figure 2: Comparison of CoT and LoT. "$\bigcirc$" denotes the thought and "→" represents the connection between two thoughts.
• Figure 3: Examples of the three types of LoT-based Oogiri games. Players are required to make surprising and creative humorous responses (blue box) to the given multimodal information e.g., images, text, or both.
• Figure 4: The overview of proposed Creative Leap-of-Thought.
• Figure 5: The LoT-oriented instruction templates.