Towards Fair and Privacy-Aware Transfer Learning for Educational Predictive Modeling: A Case Study on Retention Prediction in Community Colleges

TL;DR

This study investigates how to responsibly transfer educational predictive models for college retention across institutions under strict privacy constraints. It evaluates direct-transfer risks and several transfer-learning strategies, including sequential training and Source-Free Domain Adaptation, using data from 27 U.S. institutions across seven cohorts. The findings show meaningful performance and fairness degradation with unadapted transfers, but demonstrate that context-aware model selection, sequential training, and group-aware evaluation thresholds can improve both accuracy and equity without data sharing. The work highlights practical approaches for broader access to predictive analytics in under-resourced settings and underscores the importance of context and privacy in model training, selection, and deployment.

Abstract

Predictive analytics is widely used in learning analytics, but many resource-constrained institutions lack the capacity to develop their own models or rely on proprietary ones trained in different contexts with little transparency. Transfer learning holds promise for expanding equitable access to predictive analytics but remains underexplored due to legal and technical constraints. This paper examines transfer learning strategies for retention prediction at U.S. two-year community colleges. We envision a scenario where community colleges collaborate with each other and four-year universities to develop retention prediction models under privacy constraints and evaluate risks and improvement strategies of cross-institutional model transfer. Using administrative records from 4 research universities and 23 community colleges covering over 800,000 students across 7 cohorts, we identify performance and fairness degradation when external models are deployed locally without adaptation. Publicly available contextual information can forecast these performance drops and offer early guidance for model portability. For developers under privacy regulations, sequential training selecting institutions based on demographic similarities enhances fairness without compromising performance. For institutions lacking local data to fine-tune source models, customizing evaluation thresholds for sensitive groups outperforms standard transfer techniques in improving performance and fairness. Our findings suggest the value of transfer learning for more accessible educational predictive modeling and call for judicious use of contextual information in model training, selection, and deployment to achieve reliable and equitable model transfer.

Figures (10)

• Figure 1: Test AUC (left) and AUC Gaps (right) of pre-trained models for each target institutions. Blue circles represent the tested metrics of pre-trained direct transfer models for each target institution, while red squares indicate the tested metrics of ideal local models.
• Figure 2: Association between contextual similarity metrics and AUC Drop (left) and AUC Gap (right). A negative coefficient indicates a reduction in the AUC Drop or AUC Gap. Error bars represent 95% confidence intervals.
• Figure 3: Relationship between overall contextual similarity and the model's transferability and fairness. Blue circles represent the tested metric of pre-trained direct transfer models on each target institution.
• Figure 4: Performance distributions across all target institutions in the dataset, comparing Test AUC for three scenarios: MSTI model, the ideal local model, and the expected outcome. The histograms are normalized to have an area of 1 (density).
• Figure 5: Differences between MSTI's Test AUC and AUC Gap with expected values across target institutions.