Who Laughs with Whom? Disentangling Influential Factors in Humor Preferences across User Clusters and LLMs

TL;DR

This work investigates how humor preferences vary across user groups and how large language models (LLMs) align with or diverge from these preferences. By clustering users from the Oogiri voting dataset and fitting cluster-specific humor-factor weights with a Bradley–Terry–Luce framework, the study reveals distinct and shared humor drivers across clusters. It then elicits LLM humor preferences via funniest-response tasks, comparing them to human clusters and demonstrating that persona prompting can align LLM preferences with particular user groups. The findings highlight the importance of accounting for preference heterogeneity in humor evaluation and offer a pathway to personalized humor generation, while noting limitations related to language, modality, and data provenance.

Abstract

Humor preferences vary widely across individuals and cultures, complicating the evaluation of humor using large language models (LLMs). In this study, we model heterogeneity in humor preferences in Oogiri, a Japanese creative response game, by clustering users with voting logs and estimating cluster-specific weights over interpretable preference factors using Bradley-Terry-Luce models. We elicit preference judgments from LLMs by prompting them to select the funnier response and found that user clusters exhibit distinct preference patterns and that the LLM results can resemble those of particular clusters. Finally, we demonstrate that, by persona prompting, LLM preferences can be directed toward a specific cluster. The scripts for data collection and analysis will be released to support reproducibility.

Figures (13)

• Figure 1: Overview of humor-preference factor analysis across user clusters and LLMs. Users are clustered based on their voting history, and humor-preference factors are analyzed using the Bradley-Terry-Luce model.
• Figure 2: UMAP visualization of user clusters. Different colors represent different clusters. C0 to C6 denote the respective user clusters and the number of users in each cluster is indicated in parentheses.
• Figure 3: BTL scores of humor preference factors for each user cluster and LLM. C0 to C6 represent each user cluster. "All users" indicates the BTL scores calculated using all users without clustering. Each LLM shows the BTL scores calculated using humor preference data in the no_persona setting. Due to space limitations, only the top 10 and bottom 10 factors based on the BTL scores of "All users" are displayed here. The BTL scores for all factors are provided in Appendices \ref{['appendix:full_btl_scores_for_user_cluster_analysis']} and \ref{['appendix:full_btl_scores_for_llm_analysis']}.
• Figure 4: Pearson's correlation matrix of BTL scores between user clusters and LLMs. By comparing user clusters and LLMs, we can identify the LLMs that align with the humor preferences of specific user clusters.
• Figure 5: Pearson's correlation matrix between Gemini 3 Pro persona and user clusters, computed over the BTL scores of humor preference factors.