Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HOTVCOM: Generating Buzzworthy Comments for Videos

Yuyan Chen, Yiwen Qian, Songzhou Yan, Jiyuan Jia, Zhixu Li, Yanghua Xiao, Xiaobo Li, Ming Yang, Qingpei Guo

TL;DR

This work tackles the problem of generating hot comments for Chinese short videos by introducing HotVCom, the largest Chinese video hot-comment dataset, and the ComHeat framework that integrates visual, audio, and textual signals. ComHeat combines supervised fine-tuning, reinforcement learning with a reward model, and a knowledge-enhanced Tree-of-Thought to produce engaging, widely liked comments, guided by a comprehensive evaluation metric capturing informativeness, relevance, creativity, and user engagement. Empirical results show ComHeat outperforms diverse baselines on HotVCom and other video-comment tasks, with demonstrated cross-linguistic effectiveness on English TikTok data. The study contributes broadly useful datasets, evaluation protocols, and a scalable, multi-modal method for hot-comment generation with potential marketing and branding impact on video platforms, and points to future work in ethics, fairness, and cross-lingual expansion.

Abstract

In the era of social media video platforms, popular ``hot-comments'' play a crucial role in attracting user impressions of short-form videos, making them vital for marketing and branding purpose. However, existing research predominantly focuses on generating descriptive comments or ``danmaku'' in English, offering immediate reactions to specific video moments. Addressing this gap, our study introduces \textsc{HotVCom}, the largest Chinese video hot-comment dataset, comprising 94k diverse videos and 137 million comments. We also present the \texttt{ComHeat} framework, which synergistically integrates visual, auditory, and textual data to generate influential hot-comments on the Chinese video dataset. Empirical evaluations highlight the effectiveness of our framework, demonstrating its excellence on both the newly constructed and existing datasets.

HOTVCOM: Generating Buzzworthy Comments for Videos

TL;DR

This work tackles the problem of generating hot comments for Chinese short videos by introducing HotVCom, the largest Chinese video hot-comment dataset, and the ComHeat framework that integrates visual, audio, and textual signals. ComHeat combines supervised fine-tuning, reinforcement learning with a reward model, and a knowledge-enhanced Tree-of-Thought to produce engaging, widely liked comments, guided by a comprehensive evaluation metric capturing informativeness, relevance, creativity, and user engagement. Empirical results show ComHeat outperforms diverse baselines on HotVCom and other video-comment tasks, with demonstrated cross-linguistic effectiveness on English TikTok data. The study contributes broadly useful datasets, evaluation protocols, and a scalable, multi-modal method for hot-comment generation with potential marketing and branding impact on video platforms, and points to future work in ethics, fairness, and cross-lingual expansion.

Abstract

In the era of social media video platforms, popular ``hot-comments'' play a crucial role in attracting user impressions of short-form videos, making them vital for marketing and branding purpose. However, existing research predominantly focuses on generating descriptive comments or ``danmaku'' in English, offering immediate reactions to specific video moments. Addressing this gap, our study introduces \textsc{HotVCom}, the largest Chinese video hot-comment dataset, comprising 94k diverse videos and 137 million comments. We also present the \texttt{ComHeat} framework, which synergistically integrates visual, auditory, and textual data to generate influential hot-comments on the Chinese video dataset. Empirical evaluations highlight the effectiveness of our framework, demonstrating its excellence on both the newly constructed and existing datasets.
Paper Structure (26 sections, 13 equations, 5 figures, 27 tables)

This paper contains 26 sections, 13 equations, 5 figures, 27 tables.

Table of Contents

  1. Introduction
  2. Datasets
  3. Evaluation
  4. Methods
  5. Visual Feature Extraction
  6. Supervised Fine-Tuning
  7. Reinforcement Learning
  8. Tree-of-Thought Refining
  9. Experiments
  10. Experimental Setups
  11. Datasets, Baselines and Metrics
  12. Main Results
  13. Ablation Study
  14. Case Study
  15. Related Work
  16. ...and 11 more sections

Figures (5)

  • Figure 1: A hot comment attracts many likes and replies compared with a cold comment for a short video.
  • Figure 2: The process of constructing Chinese short video comment dataset HotVCom.
  • Figure 3: The overview of the proposed Chinese video hot-comments generation framework ComHeat.
  • Figure 4: The performance of adopting other LLMs as backbones in Chinese video comment generation.
  • Figure 5: The exploratory data analysis on HotVCom.