Knowledge Synthesis Graph: An LLM-Based Approach for Modeling Student Collaborative Discourse

TL;DR

The paper addresses the difficulty of surfacing and advancing ideas in asynchronous student discourse by introducing the Knowledge Synthesis Graph (KSG), an LLM-guided intermediate artifact that encodes Micro-ideas, Synthesis Nodes, and Epistemic Relations to visualize evolving collaborative knowledge. It proposes a three-stage pipeline to construct KSGs from social annotation data, combining iterative prompt engineering with human expert coding to preserve epistemic nuance and learner agency. Preliminary evaluations across prompts and models demonstrate feasibility and reveal how prompt structure impacts accuracy, reliability, and consistency of synthesis links, highlighting the two-level coding scheme as a key driver of stability. The work lays a technical foundation for AI-assisted, pedagogically aligned knowledge work in CSCL and points to interactive, dynamic graphs as scaffolds for ongoing student inquiry and human–AI collaboration.

Abstract

Asynchronous, text-based discourse-such as students' posts in discussion forums-is widely used to support collaborative learning. However, the distributed and evolving nature of such discourse often makes it difficult to see how ideas connect, develop, and build on one another over time. As a result, learners may struggle to recognize relationships among ideas-a process that is critical for idea advancement in productive collaborative discourse. To address this challenge, we explore how large language models (LLMs) can provide representational guidance by modeling student discourse as a Knowledge Synthesis Graph (KSG). The KSG identifies ideas from student discourse and visualizes their epistemic relationships, externalizing the current state of collaborative knowledge in a form that can support further inquiry and idea advancement. In this study, we present the design of the KSG and evaluate the LLM-based approach for constructing KSGs from authentic student discourse data. Through multi-round human-expert coding and prompt iteration, our results demonstrate the feasibility of using our approach to construct reliable KSGs across different models. This work provides a technical foundation for modeling collaborative discourse with LLMs and offers pedagogical implications for augmenting complex knowledge work in collaborative learning environments.

Figures (4)

• Figure 1: An illustrative example of the Knowledge Synthesis Graph.
• Figure 2: Technical pipeline for KSG construction.
• Figure 3: Agreement of Micro-idea labeling in stage 1.
• Figure 4: Cross-model execution rate and linking consistency in stage 3.