Representational Alignment Supports Effective Machine Teaching

TL;DR

This work investigates representational alignment as a core determinant of teaching effectiveness, introducing GRADE to controllably manipulate teacher accuracy and teacher-student representation similarity on a grid-based task. Through machine-machine and machine-human experiments, the authors derive a dyadic utility curve linking alignment, accuracy, and learning outcomes, and show that alignment can compensate for lower accuracy in some cases. Extending to classroom-scale settings, they demonstrate that class size and representational diversity moderate alignment benefits, and propose GRADE-Match to optimally assign students to teachers. The findings highlight that future AI tutors and educational systems should prioritize alignment with human learners in addition to raw task performance, informing mechanisms for personalized teaching and teacher matching. The work also provides a theoretical formulation and rich appendix detailing simulations, human experiments, and classroom simulations, enabling future replication and extension.

Abstract

A good teacher should not only be knowledgeable, but should also be able to communicate in a way that the student understands -- to share the student's representation of the world. In this work, we introduce a new controlled experimental setting, GRADE, to study pedagogy and representational alignment. We use GRADE through a series of machine-machine and machine-human teaching experiments to characterize a utility curve defining a relationship between representational alignment, teacher expertise, and student learning outcomes. We find that improved representational alignment with a student improves student learning outcomes (i.e., task accuracy), but that this effect is moderated by the size and representational diversity of the class being taught. We use these insights to design a preliminary classroom matching procedure, GRADE-Match, that optimizes the assignment of students to teachers. When designing machine teachers, our results suggest that it is important to focus not only on accuracy, but also on representational alignment with human learners.

Figures (8)

• Figure 1: Overview. A: GRADE task; teacher and student receive misaligned grids (numbers only represent re-arranged elements, participants do not see them). Teacher is shown all labels (shown as colors) and reveals one per class to the student. Teacher's error rate is controlled by mislabeling their grid. B: Our hypothesized causal model. C: Dyadic interaction between a teacher and a student. "Student-centric" teachers infer the student's representations, making them fully aligned. D: "Classroom" setting where teacher broadcasts examples to all students (who have individual differences in representations); student-centric teachers jointly optimize over all students in the class. E: "School" setting where teachers are matched with students; each student is matched with a single teacher. F: Utility curve relating teacher error rate, representational alignment, and student accuracy in simulations. G: Average accuracy and standard errors in a student-centric class as a function of class size in simulations. H: Average accuracy and standard errors across a school achieved by different matching procedures in simulations.
• Figure 2: Relating teacher error rate and representational alignment between machine teachers and human students to student accuracy. From left to right: simple-features one class per quadrant; salient-dinos one per quadrant; simple-features one class per column (6); salient-dinos one class per column (7). Format follows Figure \ref{['fig:main-schematic']}F.
• Figure 3: Schematic of teaching and representational alignment. Teachers and students have distinct representational spaces ($X, Y_s$) with some mapping between them ($T_s$). There is a true label function ($f$) that can be projected onto both the teacher and student spaces, but a teacher may not perfectly know this true label function and have their own, diverging label function ($f'$). The teacher designs curricular materials ($L_0$; a set of examples paired with labels) that are projected to each student's space ($L_s$), where each student uses them to learn a label function ($g_s$). Each student's performance ($V_s$) is then measured as the divergence between the learned label function and the hidden true label function ($T'_s(f)$).
• Figure 4: Utility curves on a $6\times6$ grid for different label structures. (Left:) 4 class underlying label-per-quadrant; (Right:) 6 class underlying label-per-column.
• Figure 5: (Left:) Group sizes from greedily incorporating the lowest performing students into the classroom of a single student-centric teacher. (Right:) Average accuracy gains (out of 1.0) in performance for students grouped with the student-centered teacher, on top of what they would have achieved from a self-centered teacher. Error bars are standard errors over $20$ seeds of student-centric teacher groupings for a sampled structured pool of students and teachers.