EmoFeedback$^2$: Reinforcement of Continuous Emotional Image Generation via LVLM-based Reward and Textual Feedback

TL;DR

This work tackles the challenge of controlling continuous emotions in image generation by introducing EmoFeedback$^2$, a generation understanding feedback reinforcement framework that leverages a fine tuned Large Vision Language Model to provide emotion aware rewards and self prompting textual feedback. The approach combines a GRPO based emotion understanding LVLM with an emotion aware reward signal to fine tune a diffusion model for improved emotional continuity, and a test time self promotion textual feedback loop to iteratively refine prompts. Key contributions include a dedicated emotion understanding LVLM trained with multi task rewards, an emotion aware reward feedback mechanism with KL regularization and a PickScore based defense against reward hacking, and a self promotion textual feedback module for adaptive prompt refinement. Extensive experiments on a custom EmoSet-118K derived dataset and cross domain evaluation on EMOTIC demonstrate state of the art performance in both emotional fidelity and image quality, with ablations confirming the value of each component and user studies validating perceptual improvements. The framework enables more reliable and controllable emotional image generation with practical implications for personalized content creation and interactive AI systems.

Abstract

Continuous emotional image generation (C-EICG) is emerging rapidly due to its ability to produce images aligned with both user descriptions and continuous emotional values. However, existing approaches lack emotional feedback from generated images, limiting the control of emotional continuity. Additionally, their simple alignment between emotions and naively generated texts fails to adaptively adjust emotional prompts according to image content, leading to insufficient emotional fidelity. To address these concerns, we propose a novel generation-understanding-feedback reinforcement paradigm (EmoFeedback$^2$) for C-EICG, which exploits the reasoning capability of the fine-tuned large vision-language model (LVLM) to provide reward and textual feedback for generating high-quality images with continuous emotions. Specifically, we introduce an emotion-aware reward feedback strategy, where the LVLM evaluates the emotional values of generated images and computes the reward against target emotions, guiding the reinforcement fine-tuning of the generative model and enhancing the emotional continuity of images. Furthermore, we design a self-promotion textual feedback framework, in which the LVLM iteratively analyzes the emotional content of generated images and adaptively produces refinement suggestions for the next-round prompt, improving the emotional fidelity with fine-grained content. Extensive experimental results demonstrate that our approach effectively generates high-quality images with the desired emotions, outperforming existing state-of-the-art methods in our custom dataset. The code and dataset will be released soon.

Figures (13)

• Figure 1: The framework of the EmoFeedback2. During training, given a neutral prompt, an emotional prompt, the V and A scores, the generative model produces a set of images. The emotion understanding model then evaluates the images to provide reward feedback. During testing, the emotional prompt is omitted due to the users' affective gap, and the model instead iteratively generates textual feedback to refine the prompts.
• Figure 2: The Emotion Understanding Model Training Process. The training image is input into the emotion understanding model to predict the V-A scores and emotion labels. The set of outputs is then fed into the designed reward functions to calculate the reward. The GRPO algorithm finally derives the advantage and loss to optimize the policy model.
• Figure 3: Qualitative comparisons with baselines under specific emotional states. Our approach demonstrates superior performance in many kinds of emotions.
• Figure 4: Qualitative comparisons with baselines under varying emotional values.
• Figure 5: Ablation study on the reward and textual feedback.