PeerGPT: Probing the Roles of LLM-based Peer Agents as Team Moderators and Participants in Children's Collaborative Learning

TL;DR

The paper investigates the dual roles of LLM-based peer agents as team moderators and participants in children's collaborative learning. It employs two design-based workshops with six 11–12-year-olds and a GPT-3.5–based peer agent to analyze effects on peer conversations through thematic analysis and quantitative measures. Findings reveal that moderator agents can structure discussions but may be ignored, while participant agents can stimulate creativity when feedback is timely, highlighting design trade-offs. The work informs design considerations for educational AI agents and points to improvements in responsiveness and nonverbal interaction alignment for future deployments.

Abstract

In children's collaborative learning, effective peer conversations can significantly enhance the quality of children's collaborative interactions. The integration of Large Language Model (LLM) agents into this setting explores their novel role as peers, assessing impacts as team moderators and participants. We invited two groups of participants to engage in a collaborative learning workshop, where they discussed and proposed conceptual solutions to a design problem. The peer conversation transcripts were analyzed using thematic analysis. We discovered that peer agents, while managing discussions effectively as team moderators, sometimes have their instructions disregarded. As participants, they foster children's creative thinking but may not consistently provide timely feedback. These findings highlight potential design improvements and considerations for peer agents in both roles.

Figures (4)

• Figure 1: Outcomes and process overview of children's collaborative learning workshops.
• Figure 2: Tasks and conversation samples of the LLM-based peer agents
• Figure 3: The themes in collaborative learning process
• Figure 4: Comparison of peer agent's participation in different stages of the workshops