Validating the Effectiveness of a Large Language Model-based Approach for Identifying Children's Development across Various Free Play Settings in Kindergarten

TL;DR

This paper addresses the challenge of assessing holistic child development during unstructured free play by combining Large Language Models with learning analytics to infer multiple developmental abilities from children’s self-narratives. The authors design a five-step technical framework, implement a relational data design, and craft structured prompts to translate narratives into Ability-Performance scores across eight domains and four play settings. Empirical evaluation with 29 children and 2,624 LLM-derived outcomes shows high reliability (>90% accuracy) for most cognitive, motor, and social abilities, and reveals setting-specific contributions to development, with empathy proving more challenging to assess. The findings demonstrate the practicality of child-centered, data-driven insights to support personalized learning and inform free-play design, while also highlighting limitations and the need for teacher validation and refinement of emotional-ability inferences.

Abstract

Free play is a fundamental aspect of early childhood education, supporting children's cognitive, social, emotional, and motor development. However, assessing children's development during free play poses significant challenges due to the unstructured and spontaneous nature of the activity. Traditional assessment methods often rely on direct observations by teachers, parents, or researchers, which may fail to capture comprehensive insights from free play and provide timely feedback to educators. This study proposes an innovative approach combining Large Language Models (LLMs) with learning analytics to analyze children's self-narratives of their play experiences. The LLM identifies developmental abilities, while performance scores across different play settings are calculated using learning analytics techniques. We collected 2,224 play narratives from 29 children in a kindergarten, covering four distinct play areas over one semester. According to the evaluation results from eight professionals, the LLM-based approach achieved high accuracy in identifying cognitive, motor, and social abilities, with accuracy exceeding 90% in most domains. Moreover, significant differences in developmental outcomes were observed across play settings, highlighting each area's unique contributions to specific abilities. These findings confirm that the proposed approach is effective in identifying children's development across various free play settings. This study demonstrates the potential of integrating LLMs and learning analytics to provide child-centered insights into developmental trajectories, offering educators valuable data to support personalized learning and enhance early childhood education practices.

Figures (3)

• Figure 1: Technical framework of the LLM-base approach
• Figure 2: Children's performance across different ability dimensions
• Figure 3: Differences in children's performance across various ability dimensions in different settings