SocialQuotes: Learning Contextual Roles of Social Media Quotes on the Web

TL;DR

This work addresses the challenge of understanding how social media quotes function within web pages by proposing a formal framework that treats embedded posts as quotes with latent roles. It introduces an eight-role taxonomy and a four-stage model of social quotation, and builds SocialQuotes, a large-scale dataset derived from Common Crawl with topic annotations and 8.3k human-annotated roles across 32.6M quotes. The authors demonstrate that a state-of-the-art LLM can infer quote roles from page context, with improvements from few-shot, chain-of-thought, self-consistency, and persistence techniques, and they analyze cross-domain distributions of platforms, domains, and roles to reveal culture- vs. reporting-oriented patterns. The dataset and methodology pave the way for cross-platform retrieval, off-API analyses, and potential generative citation, enabling richer, context-aware studies of the web's social media landscape.

Abstract

Web authors frequently embed social media to support and enrich their content, creating the potential to derive web-based, cross-platform social media representations that can enable more effective social media retrieval systems and richer scientific analyses. As step toward such capabilities, we introduce a novel language modeling framework that enables automatic annotation of roles that social media entities play in their embedded web context. Using related communication theory, we liken social media embeddings to quotes, formalize the page context as structured natural language signals, and identify a taxonomy of roles for quotes within the page context. We release SocialQuotes, a new data set built from the Common Crawl of over 32 million social quotes, 8.3k of them with crowdsourced quote annotations. Using SocialQuotes and the accompanying annotations, we provide a role classification case study, showing reasonable performance with modern-day LLMs, and exposing explainable aspects of our framework via page content ablations. We also classify a large batch of un-annotated quotes, revealing interesting cross-domain, cross-platform role distributions on the web.

Figures (6)

• Figure 1: Fictional web article with social media quotes from fictional platforms. We model social media quotation using a 4-stage procedure from communications theory haapanen2020modelling. The "Societal localization" stage is an unobserved process wherein the web author decides which roles to seek for contextual support. We introduce a framework for inferring roles from the web context.
• Figure 2: The frequency of each of the eight labels in the final dataset.
• Figure 3: The first question in the human annotation user interface. Annotators were asked if they could actually find the social quote in a current-day live rendering of the web page.
• Figure 4: The second question in the human annotation user interface. Once it was determined that the annotator could find the social quote in the page, the annotator was asked to select any number of roles they thought reflected the web author's view on the social media post. When they selected the options, short snippets appeared reminding the annotator of the role definition, which are similar to our definitions laid out in Table \ref{['tab:taxonomy']}.
• Figure 5: Proportion of times each role pair co-appeared for quote with a valid ground-truth annotation.