Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Measuring Inclusion in Interaction: Inclusion Analytics for Human-AI Collaborative Learning

Jaeyoon Choi, Nia Nixon

TL;DR

Measuring inclusion in collaborative problem solving with AI requires capturing moment-to-moment interactional dynamics. The authors propose inclusion analytics, a discourse-based framework that operationalizes participation equity, affective climate, and epistemic equity through interaction-level metrics such as $IP$, politeness uptake, and uptake-based measures, demonstrated on simulated CPS discourse and a human–AI Moon Survival task. Results show robust signals for participation balance, mixed findings for affective climate, and nuanced epistemic dynamics, highlighting both promise and current limitations of these simple, scalable metrics. This work advances process-oriented assessment of inclusion in human–AI collaboration and invites further refinement and cross-context validation to guide design and evaluation of inclusive learning environments.

Abstract

Inclusion, equity, and access are widely valued in AI and education, yet are often assessed through coarse sample descriptors or post-hoc self-reports that miss how inclusion is shaped moment by moment in collaborative problem solving (CPS). In this proof-of-concept paper, we introduce inclusion analytics, a discourse-based framework for examining inclusion as a dynamic, interactional process in CPS. We conceptualize inclusion along three complementary dimensions -- participation equity, affective climate, and epistemic equity -- and demonstrate how these constructs can be made analytically visible using scalable, interaction-level measures. Using both simulated conversations and empirical data from human-AI teaming experiments, we illustrate how inclusion analytics can surface patterns of participation, relational dynamics, and idea uptake that remain invisible to aggregate or post-hoc evaluations. This work represents an initial step toward process-oriented approaches to measuring inclusion in human-AI collaborative learning environments.

Measuring Inclusion in Interaction: Inclusion Analytics for Human-AI Collaborative Learning

TL;DR

Measuring inclusion in collaborative problem solving with AI requires capturing moment-to-moment interactional dynamics. The authors propose inclusion analytics, a discourse-based framework that operationalizes participation equity, affective climate, and epistemic equity through interaction-level metrics such as , politeness uptake, and uptake-based measures, demonstrated on simulated CPS discourse and a human–AI Moon Survival task. Results show robust signals for participation balance, mixed findings for affective climate, and nuanced epistemic dynamics, highlighting both promise and current limitations of these simple, scalable metrics. This work advances process-oriented assessment of inclusion in human–AI collaboration and invites further refinement and cross-context validation to guide design and evaluation of inclusive learning environments.

Abstract

Inclusion, equity, and access are widely valued in AI and education, yet are often assessed through coarse sample descriptors or post-hoc self-reports that miss how inclusion is shaped moment by moment in collaborative problem solving (CPS). In this proof-of-concept paper, we introduce inclusion analytics, a discourse-based framework for examining inclusion as a dynamic, interactional process in CPS. We conceptualize inclusion along three complementary dimensions -- participation equity, affective climate, and epistemic equity -- and demonstrate how these constructs can be made analytically visible using scalable, interaction-level measures. Using both simulated conversations and empirical data from human-AI teaming experiments, we illustrate how inclusion analytics can surface patterns of participation, relational dynamics, and idea uptake that remain invisible to aggregate or post-hoc evaluations. This work represents an initial step toward process-oriented approaches to measuring inclusion in human-AI collaborative learning environments.
Paper Structure (24 sections, 13 equations, 1 figure, 5 tables)

This paper contains 24 sections, 13 equations, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Inclusion Analytics: A Multi-Dimensional Framework
  3. Participation Equity
  4. Affective Climate
  5. Epistemic Equity
  6. Operationalizing Inclusion from Discourse
  7. Participation Equity Metrics
  8. Affective Climate Metrics
  9. Epistemic Equity Metrics
  10. Methods
  11. Simulated Conversations
  12. Human-AI Teaming Dataset
  13. Results
  14. Simulated Conversations
  15. Participation Equity
  16. ...and 9 more sections

Figures (1)

  • Figure 1: Average metric scores of speakers in the simulated data, with error bars showing 1 standard error of the mean. The left column shows the balanced cases, and the right column shows the imbalanced cases.