Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GASTON: Graph-Aware Social Transformer for Online Networks

Olha Wloch, Liam Hebert, Robin Cohen, Lukasz Golab

TL;DR

GASTON introduces a graph-aware transformer that grounds text and user representations in the local norms of online communities by learning trainable community embeddings through a contrastive initialization. It fuses textual semantics with structural context via a heterogeneous graph transformer operating on users, texts, and communities, pretraining with text reconstruction and edge generation losses. Across five downstream tasks on Reddit data, GASTON achieves state-of-the-art performance on several socially nuanced predictions, notably norm violations and hate speech, while ablations highlight the value of learnable, structurally informed community nodes and a capable language-model backbone. The approach offers a path toward context-aware, community-sensitive analysis of online discourse, with implications for safer and more transparent online spaces, albeit with caveats on data scale and model complexity.

Abstract

Online communities have become essential places for socialization and support, yet they also possess toxicity, echo chambers, and misinformation. Detecting this harmful content is difficult because the meaning of an online interaction stems from both what is written (textual content) and where it is posted (social norms). We propose GASTON (Graph-Aware Social Transformer for Online Networks), which learns text and user embeddings that are grounded in their local norms, providing the necessary context for downstream tasks. The heart of our solution is a contrastive initialization strategy that pretrains community embeddings based on user membership patterns, capturing a community's user base before processing any text. This allows GASTON to distinguish between communities (e.g., a support group vs. a hate group) based on who interacts there, even if they share similar vocabulary. Experiments on tasks such as stress detection, toxicity scoring, and norm violation demonstrate that the embeddings produced by GASTON outperform state-of-the-art baselines.

GASTON: Graph-Aware Social Transformer for Online Networks

TL;DR

GASTON introduces a graph-aware transformer that grounds text and user representations in the local norms of online communities by learning trainable community embeddings through a contrastive initialization. It fuses textual semantics with structural context via a heterogeneous graph transformer operating on users, texts, and communities, pretraining with text reconstruction and edge generation losses. Across five downstream tasks on Reddit data, GASTON achieves state-of-the-art performance on several socially nuanced predictions, notably norm violations and hate speech, while ablations highlight the value of learnable, structurally informed community nodes and a capable language-model backbone. The approach offers a path toward context-aware, community-sensitive analysis of online discourse, with implications for safer and more transparent online spaces, albeit with caveats on data scale and model complexity.

Abstract

Online communities have become essential places for socialization and support, yet they also possess toxicity, echo chambers, and misinformation. Detecting this harmful content is difficult because the meaning of an online interaction stems from both what is written (textual content) and where it is posted (social norms). We propose GASTON (Graph-Aware Social Transformer for Online Networks), which learns text and user embeddings that are grounded in their local norms, providing the necessary context for downstream tasks. The heart of our solution is a contrastive initialization strategy that pretrains community embeddings based on user membership patterns, capturing a community's user base before processing any text. This allows GASTON to distinguish between communities (e.g., a support group vs. a hate group) based on who interacts there, even if they share similar vocabulary. Experiments on tasks such as stress detection, toxicity scoring, and norm violation demonstrate that the embeddings produced by GASTON outperform state-of-the-art baselines.
Paper Structure (27 sections, 6 equations, 5 figures, 3 tables)

This paper contains 27 sections, 6 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Our Solution
  4. Excecutive Summary
  5. Initializing Community Node Embeddings
  6. Initializing User Node Embeddings
  7. Node Type Projections
  8. Graph Transformer Layers
  9. Pretraining Setup and Tasks
  10. Pretraining Metrics
  11. Experiments
  12. Pretraining Data
  13. Fine-Tuning for Downstream Tasks
  14. Evaluation Metrics
  15. Baselines
  16. ...and 12 more sections

Figures (5)

  • Figure 1: A fragment of the Reddit social platform
  • Figure 2: Node and edge types used in GASTON
  • Figure 3: An architectural comparison of GASTON with prior work
  • Figure 4: The GASTON pretraining architecture showing text reconstruction and edge prediction tasks.
  • Figure 5: Visualizing GASTON's community embeddings