Advancing credit mobility through stakeholder-informed AI design and adoption

TL;DR

This study tackles credit mobility barriers by applying stakeholder-informed AI to SUNY course articulation, starting with surveys of staff and faculty to shape design requirements. It introduces Shared Space Alignment (SSA), a supervised embedding alignment that, when combined with OpenAI-style embeddings and Course2vec signals, achieves recall@1 of $0.764$ and recall@5 of $0.928$, far surpassing prior work. Using an AUC-ROC threshold of $0.547$ and five-fold cross-validation, the approach reduces CIP dispersion (via the metric $r_g^{(p)}$) and surfaces $2{,}787{,}526$ additional articulation candidates, with an adoption projection of about $1{,}706{,}802$ new credit-mobility opportunities at a conservative $61.23 ext{ }%$ uptake. Findings suggest that aligning AI design with stakeholder expectations can expand transfer pathways, inform policy, and reduce credit loss, offering a scalable blueprint for AI-assisted articulation in higher education.

Abstract

Transferring from a 2-year to a 4-year college is crucial for socioeconomic mobility, yet students often face challenges ensuring their credits are fully recognized, leading to delays in their academic progress and unexpected costs. Determining whether courses at different institutions are equivalent (i.e., articulation) is essential for successful credit transfer, as it minimizes unused credits and increases the likelihood of bachelor's degree completion. However, establishing articulation agreements remains time- and resource-intensive, as all candidate articulations are reviewed manually. Although recent efforts have explored the use of artificial intelligence to support this work, its use in articulation practice remains limited. Given these challenges and the need for scalable support, this study applies artificial intelligence to suggest articulations between institutions in collaboration with the State University of New York system, one of the largest systems of higher education in the US. To develop our methodology, we first surveyed articulation staff and faculty to assess adoption rates of baseline algorithmic recommendations and gather feedback on perceptions and concerns about these recommendations. Building on these insights, we developed a supervised alignment method that addresses superficial matching and institutional biases in catalog descriptions, achieving a 5.5-fold improvement in accuracy over previous methods. Based on articulation predictions of this method and a 61% average surveyed adoption rate among faculty and staff, these findings project a 12-fold increase in valid credit mobility opportunities that would otherwise remain unrealized. This study suggests that stakeholder-informed design of AI in higher education administration can expand student credit mobility and help reshape current institutional decision-making in course articulation.

Figures (8)

• Figure 1: Overview of the study workflow. This figure illustrate the end-to-end workflow of the study, beginning with generating articulation recommendations using SBERT embeddings, followed by faculty and staff evaluations of the recommendations. Feedback from the evaluation informs advancements to the method. The improved approach is used for articulation expansion, and we project adoption rates for these expanded articulation candidates based on observed faculty and staff acceptance patterns.
• Figure 2: Overview of data sources. Geographic distribution of institutions and dataset descriptives. Dark circles represent colleges included in this study, while white circles indicate those not included.
• Figure 3: Methodological framework for AI-assistive course equivalency process. This figure provides a high-level overview of the methodology and the task objectives. The process begins with the vectorization of course titles and descriptions using a readily available state-of-the-art large language model. We then refined these vectors through a shared space alignment technique, leveraging pre-established course articulations. The resulting vectors were validated using faculty-established articulations and then applied to system-specific tasks (i.e., course articulation at SUNY), with the outcomes were evaluated accordingly.
• Figure 4: The encoder and decoder process of SSA. A course vector $x_i$ from college $i$ is encoded into a shared space using the transformation matrix $M_i$.The resulting vector, $x_i \cdot M_i$, is then decoded into the vector space of college $j$ using the transpose of the transformation matrix $M_j^{T}$. The error is calculated between this predicted vector, $x_i \cdot M_i \cdot M_j^{T}$, and the true equivalent course vector $x_j$ from college $j$. This error is minimized through iterative updates of the transformation matrices $M$ using backpropagation rumelhart1986learning. After optimization, the outputs of this algorithm are blue encoded shared space vectors, $x_i \cdot M_i$.
• Figure 5: An example of similarity thresholding for course articulation expansion, based on SSA OpenAI embeddings for SUNY courses. A. Relationship between threshold values and TPR, FNR, TNR, and FPR. The optimal threshold is selected to balance maximizing the True Positive Rate (TPR) while minimizing the False Positive Rate (FPR). Note that TPR = 1 - FNR and TNR = 1 - FPR. B. Distribution of cosine similarities between course pairs: pseudo-negatives vs. pre-established articulations. Existing articulation pairs have higher cosine similarities (average 0.869), while pseudo-negative pairs have lower similarities (average 0.321).