CBVS: A Large-Scale Chinese Image-Text Benchmark for Real-World Short Video Search Scenarios

TL;DR

CBVS addresses the scarcity of large-scale Chinese cover-image data for real-world short video search by releasing CBVS-5M/10M (video title–cover with OCR) and CBVS-20K (manual query–cover) datasets. It introduces UniCLIP, an OCR-free training framework with presence-guided and semantic-guided encoders that leverage cover-text semantics during training while avoiding OCR in inference, optimized with losses $L_{ITC}$, $L_{IC}$, and $L_{ITM}$. On CBVS-20K, UniCLIP achieves state-of-the-art image-text matching, with ablations confirming the additive value of both encoders; fine-tuning on CBVS-5M/10M further boosts recall and ranking metrics. The dataset and method enable robust cross-modal matching in Chinese short-video search and have already been deployed in Tencent’s online video search, illustrating practical impact and broad applicability to multi-modal fusion tasks.

Abstract

Vision-Language Models pre-trained on large-scale image-text datasets have shown superior performance in downstream tasks such as image retrieval. Most of the images for pre-training are presented in the form of open domain common-sense visual elements. Differently, video covers in short video search scenarios are presented as user-originated contents that provide important visual summaries of videos. In addition, a portion of the video covers come with manually designed cover texts that provide semantic complements. In order to fill in the gaps in short video cover data, we establish the first large-scale cover-text benchmark for Chinese short video search scenarios. Specifically, we release two large-scale datasets CBVS-5M/10M to provide short video covers, and the manual fine-labeling dataset CBVS-20K to provide real user queries, which serves as an image-text benchmark test in the Chinese short video search field. To integrate the semantics of cover text in the case of modality missing, we propose UniCLIP where cover texts play a guiding role during training, however are not relied upon by inference. Extensive evaluation on CBVS-20K demonstrates the excellent performance of our proposal. UniCLIP has been deployed to Tencent's online video search systems with hundreds of millions of visits and achieved significant gains. The dataset and code are available at https://github.com/QQBrowserVideoSearch/CBVS-UniCLIP.

Figures (3)

• Figure 1: Morphological differences between open domain images and short video cover images.
• Figure 2: Top: Presentation of CBVS-20K data. Bottom: Presentation of CBVS-5M/10M data.
• Figure 3: Model structure of UniCLIP. When the model performs inference, only the green area works.