Who Said What WSW 2.0? Enhanced Automated Analysis of Preschool Classroom Speech

TL;DR

WSW 2.0 tackles the challenge of scalable automated analysis of preschool classrooms by jointly identifying who is speaking and what is being said, using a wav2vec2‑based speaker classifier (ESC) and Whisper ASR. The authors validate the pipeline against expert annotations on 210 minutes and demonstrate reliability across a broad set of language features, reporting accuracy 0.846, weighted F1 0.845, and WERs of 0.119 (teacher) and 0.238 (child). They further scale the approach to 1,592 hours of classroom audio across two years, processing hundreds of thousands of utterances and enabling robust measurement of teacher and child language measures such as MLU and lexical diversity. The results indicate that automated WSW 2.0 can support large‑scale educational research and intervention planning by linking teacher language input to child language development.

Abstract

This paper introduces an automated framework WSW2.0 for analyzing vocal interactions in preschool classrooms, enhancing both accuracy and scalability through the integration of wav2vec2-based speaker classification and Whisper (large-v2 and large-v3) speech transcription. A total of 235 minutes of audio recordings (160 minutes from 12 children and 75 minutes from 5 teachers), were used to compare system outputs to expert human annotations. WSW2.0 achieves a weighted F1 score of .845, accuracy of .846, and an error-corrected kappa of .672 for speaker classification (child vs. teacher). Transcription quality is moderate to high with word error rates of .119 for teachers and .238 for children. WSW2.0 exhibits relatively high absolute agreement intraclass correlations (ICC) with expert transcriptions for a range of classroom language features. These include teacher and child mean utterance length, lexical diversity, question asking, and responses to questions and other utterances, which show absolute agreement intraclass correlations between .64 and .98. To establish scalability, we apply the framework to an extensive dataset spanning two years and over 1,592 hours of classroom audio recordings, demonstrating the framework's robustness for broad real-world applications. These findings highlight the potential of deep learning and natural language processing techniques to revolutionize educational research by providing accurate measures of key features of preschool classroom speech, ultimately guiding more effective intervention strategies and supporting early childhood language development.

Figures (2)

• Figure 1: Workflow. Whisper is automated speech-to-text (transcription) software. The ESC(wav2vec2) provides automated speaker classification (teacher versus child). The human expert provides both speaker classification and transcription. Whisper transcription is used to synchronize the ESC(wav2vec2) and expert speaker classification.
• Figure 2: Cross-classification confusion matrix between Whisper large-v2 + ESC(wav2vec2) and the expert. The ESC(wav2vec2) produced the classification (teacher versus child) while Whisper transcription was used to align results with the expert transcription. Teacher utterances were classified more accurately than child utterances (see TABLE \ref{['tab:performance_metrics']}, Overall, for statistical descriptions)