The Value of AI-Generated Metadata for UGC Platforms: Evidence from a Large-scale Field Experiment
Xinyi Zhang, Chenshuo Sun, Renyu Zhang, Khim-Yong Goh
TL;DR
This study asks whether AI-generated video metadata provides value on UGC platforms where metadata is often sparse. Using a large-scale randomized field experiment on a major Asian short-video platform, the authors show that providing AI-generated titles increases content consumption by improving user-content matching, with larger effects when producers adopt or co-create the titles. The mechanism is supported by higher predictive accuracy (AUC) in the recommender system and reduced metadata sparsity, while results also reveal that AI-generated titles can underperform human-generated ones unless combined with substantial human revision. The findings offer practical guidance for platform operators on deploying AI-generated metadata and highlight the benefits of human-AI collaboration to maximize lexical richness and engagement.
Abstract
AI-generated content (AIGC), such as advertisement copy, product descriptions, and social media posts, is becoming ubiquitous in business practices. However, the value of AI-generated metadata, such as titles, remains unclear on user-generated content (UGC) platforms. To address this gap, we conducted a large-scale field experiment on a leading short-video platform in Asia to provide about 1 million users access to AI-generated titles for their uploaded videos. Our findings show that the provision of AI-generated titles significantly boosted content consumption, increasing valid watches by 1.6% and watch duration by 0.9%. When producers adopted these titles, these increases jumped to 7.1% and 4.1%, respectively. This viewership-boost effect was largely attributed to the use of this generative AI (GAI) tool increasing the likelihood of videos having a title by 41.4%. The effect was more pronounced for groups more affected by metadata sparsity. Mechanism analysis revealed that AI-generated metadata improved user-video matching accuracy in the platform's recommender system. Interestingly, for a video for which the producer would have posted a title anyway, adopting the AI-generated title decreased its viewership on average, implying that AI-generated titles may be of lower quality than human-generated ones. However, when producers chose to co-create with GAI and significantly revised the AI-generated titles, the videos outperformed their counterparts with either fully AI-generated or human-generated titles, showcasing the benefits of human-AI co-creation. This study highlights the value of AI-generated metadata and human-AI metadata co-creation in enhancing user-content matching and content consumption for UGC platforms.