Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding Virality: A Rubric based Vision-Language Model Framework for Short-Form Edutainment Evaluation

Arnav Gupta, Gurekas Singh Sahney, Hardik Rathi, Abhishek Chandwani, Ishaan Gupta, Pratik Narang, Dhruv Kumar

TL;DR

This work tackles the challenge of evaluating short-form edutainment by linking audiovisual features to human engagement through Vision-Language Models. It introduces a large YouTube Shorts dataset and a data-driven pipeline that extracts unsupervised multimodal features, clusters them into interpretable factors, and learns cluster weights to predict engagement with SHAP-based explanations. The approach achieves strong alignment with real engagement (Spearman 0.71, R^2 0.61) and demonstrates cross-domain generalization, while offering transparent, per-cluster insights. A VLM-as-a-Judge mechanism enables scalable, interpretable scoring for unseen videos, advancing robust, explainable video evaluation for content recommendation and comparison in edutainment contexts.

Abstract

Evaluating short-form video content requires moving beyond surface-level quality metrics toward human-aligned, multimodal reasoning. While existing frameworks like VideoScore-2 assess visual and semantic fidelity, they do not capture how specific audiovisual attributes drive real audience engagement. In this work, we propose a data-driven evaluation framework that uses Vision-Language Models (VLMs) to extract unsupervised audiovisual features, clusters them into interpretable factors, and trains a regression-based evaluator to predict engagement on short-form edutainment videos. Our curated YouTube Shorts dataset enables systematic analysis of how VLM-derived features relate to human engagement behavior. Experiments show strong correlations between predicted and actual engagement, demonstrating that our lightweight, feature-based evaluator provides interpretable and scalable assessments compared to traditional metrics (e.g., SSIM, FID). By grounding evaluation in both multimodal feature importance and human-centered engagement signals, our approach advances toward robust and explainable video understanding.

Understanding Virality: A Rubric based Vision-Language Model Framework for Short-Form Edutainment Evaluation

TL;DR

This work tackles the challenge of evaluating short-form edutainment by linking audiovisual features to human engagement through Vision-Language Models. It introduces a large YouTube Shorts dataset and a data-driven pipeline that extracts unsupervised multimodal features, clusters them into interpretable factors, and learns cluster weights to predict engagement with SHAP-based explanations. The approach achieves strong alignment with real engagement (Spearman 0.71, R^2 0.61) and demonstrates cross-domain generalization, while offering transparent, per-cluster insights. A VLM-as-a-Judge mechanism enables scalable, interpretable scoring for unseen videos, advancing robust, explainable video evaluation for content recommendation and comparison in edutainment contexts.

Abstract

Evaluating short-form video content requires moving beyond surface-level quality metrics toward human-aligned, multimodal reasoning. While existing frameworks like VideoScore-2 assess visual and semantic fidelity, they do not capture how specific audiovisual attributes drive real audience engagement. In this work, we propose a data-driven evaluation framework that uses Vision-Language Models (VLMs) to extract unsupervised audiovisual features, clusters them into interpretable factors, and trains a regression-based evaluator to predict engagement on short-form edutainment videos. Our curated YouTube Shorts dataset enables systematic analysis of how VLM-derived features relate to human engagement behavior. Experiments show strong correlations between predicted and actual engagement, demonstrating that our lightweight, feature-based evaluator provides interpretable and scalable assessments compared to traditional metrics (e.g., SSIM, FID). By grounding evaluation in both multimodal feature importance and human-centered engagement signals, our approach advances toward robust and explainable video understanding.
Paper Structure (30 sections, 2 equations, 4 figures, 5 tables)

This paper contains 30 sections, 2 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dataset Creation, Evaluation Environment & Rubric Design
  4. Multimodal Reasoning and Engagement Understanding
  5. Multimodal Reasoning, Reward Modelling and Persona-Based Evaluation
  6. Methodology
  7. Problem Setup and Dataset
  8. Overall Pipeline
  9. Audiovisual Feature Extraction
  10. Feature Representation and Analysis
  11. Engagement Modeling via Regression
  12. Evaluator Construction
  13. Implementation and Training Details
  14. Evaluation & Results
  15. Quantitative Performance and Baseline Comparison
  16. ...and 15 more sections

Figures (4)

  • Figure 1: Dataset Distribution Graphic: Analysis of views, duration, and categories.
  • Figure 2: Overall pipeline: audiovisual feature extraction, clustering, regression-based feature importance modeling, and weighted engagement prediction.
  • Figure 3: Rubric design framework for persona-based evaluation pipeline (reserved for future work).
  • Figure 4: Legacy rubric used during early prototyping, included in full resolution for reference. This rubric was later deprecated in favor of the feature-based approach described in Section 3.