Content Significance Distribution of Sub-Text Blocks in Articles and Its Application to Article-Organization Assessment

TL;DR

This work defines Content Significance Distribution (CSD) for sub-text blocks (CSD-1) and CSD for sentence locations (CSD-2) to quantify how salient portions of an article are and where they occur. It employs contextual embeddings from SentenceTransformer and MoverScore to measure block-article similarity, and introduces a scalable approximation to mitigate combinatorial explosion, revealing consistent CSD-1 patterns across article types and linking CSD-1 to a beta-distribution-based transform. The authors show that CSD-1 features can reliably predict intrinsic article organization with high accuracy on student essays using an SVC, and they characterize CSD-2 patterns that differentiate article types. Collectively, this establishes a quantitative, scalable framework for analyzing article structure with potential extensions to automated summarization and type-aware sentence ranking.

Abstract

We explore how to capture the significance of a sub-text block in an article and how it may be used for text mining tasks. A sub-text block is a sub-sequence of sentences in the article. We formulate the notion of content significance distribution (CSD) of sub-text blocks, referred to as CSD of the first kind and denoted by CSD-1. In particular, we leverage Hugging Face's SentenceTransformer to generate contextual sentence embeddings, and use MoverScore over text embeddings to measure how similar a sub-text block is to the entire text. To overcome the exponential blowup on the number of sub-text blocks, we present an approximation algorithm and show that the approximated CSD-1 is almost identical to the exact CSD-1. Under this approximation, we show that the average and median CSD-1's for news, scholarly research, argument, and narrative articles share the same pattern. We also show that under a certain linear transformation, the complement of the cumulative distribution function of the beta distribution with certain values of $α$ and $β$ resembles a CSD-1 curve. We then use CSD-1's to extract linguistic features to train an SVC classifier for assessing how well an article is organized. Through experiments, we show that this method achieves high accuracy for assessing student essays. Moreover, we study CSD of sentence locations, referred to as CSD of the second kind and denoted by CSD-2, and show that average CSD-2's for different types of articles possess distinctive patterns, which either conform common perceptions of article structures or provide rectification with minor deviation.

Figures (7)

• Figure 1: Average and median approximated CSD-1 for articles of each type, where the x-axis is the normalized sequence of text blocks in ascending order of MoverScores compared with the article itself, and the y-axis is the MoverScores
• Figure 2: CSD-1 segments
• Figure 3: Average CSD-1's with $c = 0.3$ for (a) articles of random sentences and (b) articles of similar sentences, with the same x-axis and y-axis as those in Figure \ref{['figure:2']}
• Figure 4: The curve of $C_x(0.4,0.3)$, which spans the entire y-axis from 0 to 1
• Figure 5: The curves resemble, respectively, (a) the average and median curves for news articles in Figure \ref{['figure:2']}(a), (b) the average curve for random sentences in Figure \ref{['figure:3']}(a), and (c) the average curve for similar sentences in Figure \ref{['figure:3']}(b)