CCPM: A Chinese Classical Poetry Matching Dataset

TL;DR

CCPM introduces a semantics-focused benchmark for Chinese classical poetry by framing a poem matching task: given the modern Chinese translation of a line, identify the correct original line from three distractors. The dataset builds from 31K bilingual parallel lines down to 27,218 high-quality pairs through line-level segmentation and filtering, with negatives retrieved via a BERT-based similarity model and LSH, plus a re-ranking strategy. Baseline experiments with two BERT-based approaches show strong performance (84.96% vs 82.60%), indicating meaningful semantic signals can be captured for poetic language. This dataset provides a public resource to advance deep semantic understanding and generation for Chinese classical poetry, bridging classical language with contemporary translation.

Abstract

Poetry is one of the most important art forms of human languages. Recently many studies have focused on incorporating some linguistic features of poetry, such as style and sentiment, into its understanding or generation system. However, there is no focus on understanding or evaluating the semantics of poetry. Therefore, we propose a novel task to assess a model's semantic understanding of poetry by poem matching. Specifically, this task requires the model to select one line of Chinese classical poetry among four candidates according to the modern Chinese translation of a line of poetry. To construct this dataset, we first obtain a set of parallel data of Chinese classical poetry and modern Chinese translation. Then we retrieve similar lines of poetry with the lines in a poetry corpus as negative choices. We name the dataset Chinese Classical Poetry Matching Dataset (CCPM) and release it at https://github.com/THUNLP-AIPoet/CCPM. We hope this dataset can further enhance the study on incorporating deep semantics into the understanding and generation system of Chinese classical poetry. We also preliminarily run two variants of BERT on this dataset as the baselines for this dataset.

Figures (1)

• Figure 1: The major process of the dataset construction. We first collect bilingual translation pairs and retrieve the most similar candidates from the poetry corpus as confusing choices