Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ConvoLearn: A Dataset of Constructivist Tutor-Student Dialogue

Mayank Sharma, Roy Pea, Hari Subramonyam

TL;DR

This work addresses the pedagogy gaps of LLM tutors by introducing ConvoLearn, a semi-synthetic dataset grounded in a knowledge-building framework with six dimensions of dialogic discourse. It operationalizes 21 subdimensions across middle-school Earth Science to create 1,250 high-quality tutor–student dialogues, then applies parameter-efficient fine-tuning (QLoRA) to open LLMs. Intrinsic and extrinsic evaluations, including a teacher study with 31 raters, show that a fine-tuned Mistral-7B significantly outperforms its base version and a leading proprietary model in knowledge-building behavior and perceived effectiveness. The dataset and methodology offer a scalable pathway to develop, evaluate, and iterate dialogic AI tutors aligned with constructivist learning goals, with potential impact on engagement, reasoning, and equity in K–12 science education.

Abstract

In educational applications, LLMs exhibit several fundamental pedagogical limitations, such as their tendency to reveal solutions rather than support dialogic learning. We introduce ConvoLearn (https://huggingface.co/datasets/masharma/convolearn ), a dataset grounded in knowledge building theory that operationalizes six core pedagogical dimensions: cognitive engagement, formative assessment, accountability, cultural responsiveness, metacognition, and power dynamics. We construct a semi-synthetic dataset of 1250 tutor-student dialogues (20 turns each) in middle school Earth Science through controlled interactions between human teachers and a simulated student. Using QLoRA, we demonstrate that training on this dataset meaningfully shifts LLM behavior toward knowledge-building strategies. Human evaluation by 31 teachers shows our fine-tuned Mistral 7B (M = 4.10, SD = 1.03) significantly outperforms both its base version (M = 2.59, SD = 1.11) and Claude Sonnet 4.5 (M = 2.87, SD = 1.29) overall. This work establishes a potential framework to guide future development and evaluation of constructivist AI tutors.

ConvoLearn: A Dataset of Constructivist Tutor-Student Dialogue

TL;DR

This work addresses the pedagogy gaps of LLM tutors by introducing ConvoLearn, a semi-synthetic dataset grounded in a knowledge-building framework with six dimensions of dialogic discourse. It operationalizes 21 subdimensions across middle-school Earth Science to create 1,250 high-quality tutor–student dialogues, then applies parameter-efficient fine-tuning (QLoRA) to open LLMs. Intrinsic and extrinsic evaluations, including a teacher study with 31 raters, show that a fine-tuned Mistral-7B significantly outperforms its base version and a leading proprietary model in knowledge-building behavior and perceived effectiveness. The dataset and methodology offer a scalable pathway to develop, evaluate, and iterate dialogic AI tutors aligned with constructivist learning goals, with potential impact on engagement, reasoning, and equity in K–12 science education.

Abstract

In educational applications, LLMs exhibit several fundamental pedagogical limitations, such as their tendency to reveal solutions rather than support dialogic learning. We introduce ConvoLearn (https://huggingface.co/datasets/masharma/convolearn ), a dataset grounded in knowledge building theory that operationalizes six core pedagogical dimensions: cognitive engagement, formative assessment, accountability, cultural responsiveness, metacognition, and power dynamics. We construct a semi-synthetic dataset of 1250 tutor-student dialogues (20 turns each) in middle school Earth Science through controlled interactions between human teachers and a simulated student. Using QLoRA, we demonstrate that training on this dataset meaningfully shifts LLM behavior toward knowledge-building strategies. Human evaluation by 31 teachers shows our fine-tuned Mistral 7B (M = 4.10, SD = 1.03) significantly outperforms both its base version (M = 2.59, SD = 1.11) and Claude Sonnet 4.5 (M = 2.87, SD = 1.29) overall. This work establishes a potential framework to guide future development and evaluation of constructivist AI tutors.
Paper Structure (51 sections, 3 figures, 3 tables)

This paper contains 51 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Prior Work
  3. Pedagogical Limitations of LLM Tutors
  4. Approaches to Pedagogically Align LLMs
  5. Benchmarks and Evaluation Frameworks
  6. Remaining Gaps in Pedagogy-Driven LLM Design
  7. Knowledge-Building Framework
  8. Our Paper
  9. Dataset.
  10. Fine-tuning.
  11. Evaluation.
  12. Data Collection
  13. Question Selection
  14. Feasibility Pilots
  15. Tutoring Interface Development
  16. ...and 36 more sections

Figures (3)

  • Figure 1: Multi-step pipeline for data collection, annotation and filtering.
  • Figure 2: Intrinsic and extrinsic evaluation pipelines used to assess fine-tuned models.
  • Figure 3: Mean teacher ratings (1--5) by model and pedagogical dimension (Overall)