Investigating Students' Preferences for AI Roles in Mathematical Modelling: Evidence from a Randomized Controlled Trial

TL;DR

The paper investigates how design thinking (DT), computational thinking (CT), and mathematical modelling self-efficacy (MMSE) relate and how learners prefer different AI roles during mathematical modelling (MM). Using a within-subject randomized trial with five AI personas (Tutor, TA, Peer, Struggling Student, Excellent Student) plus baseline tasks, the study combines quantitative analyses (e.g., Pearson correlations, PLS-SEM with $R^2 = 0.808$, $f^2$ effects) and qualitative feedback to reveal that DT and CT are significant predictors of MMSE, and that learners favor competent AI roles while disfavoring the Struggling Student role. A key finding is that higher DT predicts a stronger preference for the Teaching Assistant role, while CT and MMSE show more limited influence on role preferences. The results inform adaptive, learner-centered AI designs for MM education, emphasizing clear guidance and human–AI collaboration practices, and they propose a theoretical framework for MMSE in AI-supported MM. Practical implications include designing AI tutors and partner roles that provide structured guidance without overloading learners, and incorporating feedback and reflection mechanisms to sustain DSL and CT development in MM tasks.

Abstract

Mathematical modelling (MM) is a key competency for solving complex real-world problems, yet many students struggle with abstraction, representation, and iterative reasoning. Artificial intelligence (AI) has been proposed as a support for higher-order thinking, but its role in MM education is still underexplored. This study examines the relationships among students' design thinking (DT), computational thinking (CT), and mathematical modelling self-efficacy (MMSE), and investigates their preferences for different AI roles during the modelling process. Using a randomized controlled trial, we identify significant connections among DT, CT, and MMSE, and reveal distinct patterns in students' preferred AI roles, including AI as a tutor (providing explanations and feedback), AI as a tool (assisting with calculations and representations), AI as a collaborator (suggesting strategies and co-creating models), and AI as a peer (offering encouragement and fostering reflection). Differences across learner profiles highlight how students' dispositions shape their expectations for AI. These findings advance understanding of AI-supported MM and provide design implications for adaptive, learner-centered systems.

Figures (8)

• Figure 1: Intercorrelations among scale subdimensions. Pearson correlation coefficients (r) are presented, with asterisks indicating statistical significance: *$p < .05$, **$p < .01$, ***$p < .001$. The color gradient reflects the magnitude and direction of correlations, ranging from strong negative (blue) to strong positive (red).
• Figure 2: Structural equation model illustrating the relationships between Computational Thinking (CT), Design Thinking (DT), and Mathematical Modelling Self-efficacy (MM). The numbers represent standardized path coefficients, with asterisks indicating statistical significance (*$p < .05$, **$p < .01$, ***$p < .001$).
• Figure 3: A Participant preferences for AI role personas. Panel A displays mean ranking scores with standard errors for each AI role, where lower scores indicate greater preference. Brackets with asterisks denote statistically significant pairwise differences (** p < .01, *** p < .001, Bonferroni corrected). Panel B shows the distribution of individual student ratings (1 = best, 5 = worst) through box plots, with boxes representing interquartile ranges, red horizontal lines indicating medians, whiskers extending to minimum and maximum values, and circles representing outliers.
• Figure 4: Perceived effectiveness of AI roles across three key dimensions (*$p < .05$, **$p < .01$, ***$p < .001$).
• Figure 5: Comparison of AI role evaluations between high-scoring and low-scoring computational thinking groups. (A) Mean rankings of AI roles (lower = better performance). (B) Distribution of individual rankings across participants. (C) Histogram of total computational thinking scores used to define the two groups ($n=13$ each). (D) Normalized preference radar chart showing group-level performance across AI roles (higher = stronger preference).