From Perception to Punchline: Empowering VLM with the Art of In-the-wild Meme
Xueyan Li, Yingyi Xue, Mengjie Jiang, Qingzi Zhu, Yazhe Niu
TL;DR
This paper tackles open-ended meme generation by introducing HUMOR, a framework that combines hierarchical multi-path Chain-of-Thought reasoning with group-wise human preferences and stable policy optimization. It formalizes meme generation as a group-aware problem with a latent humor function per template group, and provides theoretical guarantees that exploration followed by anchoring preserves a lower humor bound. A reward-modeling module learns within-group preferences and drives updates via Group-wise Relative Policy Optimization (GRPO), with additional auxiliary rewards to stabilize reasoning paths. Empirical results show HUMOR improves meme humor, readability, and originality while maintaining reliability across base models and unseen templates, highlighting its potential as a general paradigm for human-aligned, open-ended multimodal generation. The work demonstrates that hierarchical CoT plus group-wise, rank-consistent preferences enables robust, context-sensitive humor generation and suggests broad applicability beyond memes to other subjective multimodal tasks.
Abstract
Generating humorous memes is a challenging multimodal task that moves beyond direct image-to-caption supervision. It requires a nuanced reasoning over visual content, contextual cues, and subjective humor. To bridge this gap between visual perception and humorous punchline creation, we propose HUMOR}, a novel framework that guides VLMs through hierarchical reasoning and aligns them with group-wise human preferences. First, HUMOR employs a hierarchical, multi-path Chain-of-Thought (CoT): the model begins by identifying a template-level intent, then explores diverse reasoning paths under different contexts, and finally anchors onto a high-quality, context-specific path. This CoT supervision, which traces back from ground-truth captions, enhances reasoning diversity. We further analyze that this multi-path exploration with anchoring maintains a high expected humor quality, under the practical condition that high-quality paths retain significant probability mass. Second, to capture subjective humor, we train a pairwise reward model that operates within groups of memes sharing the same template. Following established theory, this approach ensures a consistent and robust proxy for human preference, even with subjective and noisy labels. The reward model then enables a group-wise reinforcement learning optimization, guaranteeing providing a theoretical guarantee for monotonic improvement within the trust region. Extensive experiments show that HUMOR empowers various VLMs with superior reasoning diversity, more reliable preference alignment, and higher overall meme quality. Beyond memes, our work presents a general training paradigm for open-ended, human-aligned multimodal generation, where success is guided by comparative judgment within coherent output group.