Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Designing and Evaluating an Educational Recommender System with Different Levels of User Control

Qurat Ul Ain, Mohamed Amine Chatti, William Kana Tsoplefack, Rawaa Alatrash, Shoeb Joarder

TL;DR

This paper investigates how providing varying levels of user control in an educational recommender system affects user perceptions and key goals. It implements an interactive ERS in the CourseMapper MOOC platform enabling input, process, and output control and evaluates it with $N=30$ participants. Results show that user control yields positive perceptions, with strong links to transparency and moderate links to trust and satisfaction; transparency and trust interact with satisfaction in nuanced ways. The work highlights the importance of controllability for transparency and provides directions for future design and evaluation, including exploring optimal control levels and qualitative insights.

Abstract

Educational recommender systems (ERSs) play a crucial role in personalizing learning experiences and enhancing educational outcomes by providing recommendations of personalized resources and activities to learners, tailored to their individual learning needs. However, their effectiveness is often diminished by insufficient user control and limited transparency. To address these challenges, in this paper, we present the systematic design and evaluation of an interactive ERS, in which we introduce different levels of user control. Concretely, we introduce user control around the input (i.e., user profile), process (i.e., recommendation algorithm), and output (i.e., recommendations) of the ERS. To evaluate our system, we conducted an online user study (N=30) to explore the impact of user control on users' perceptions of the ERS in terms of several important user-centric aspects. Moreover, we investigated the effects of user control on multiple recommendation goals, namely transparency, trust, and satisfaction, as well as the interactions between these goals. Our results demonstrate the positive impact of user control on user perceived benefits of the ERS. Moreover, our study shows that user control strongly correlates with transparency and moderately correlates with trust and satisfaction. In terms of interaction between these goals, our results reveal that transparency moderately correlates and trust strongly correlates with satisfaction. Whereas, transparency and trust stand out as less correlated with each other.

Designing and Evaluating an Educational Recommender System with Different Levels of User Control

TL;DR

This paper investigates how providing varying levels of user control in an educational recommender system affects user perceptions and key goals. It implements an interactive ERS in the CourseMapper MOOC platform enabling input, process, and output control and evaluates it with participants. Results show that user control yields positive perceptions, with strong links to transparency and moderate links to trust and satisfaction; transparency and trust interact with satisfaction in nuanced ways. The work highlights the importance of controllability for transparency and provides directions for future design and evaluation, including exploring optimal control levels and qualitative insights.

Abstract

Educational recommender systems (ERSs) play a crucial role in personalizing learning experiences and enhancing educational outcomes by providing recommendations of personalized resources and activities to learners, tailored to their individual learning needs. However, their effectiveness is often diminished by insufficient user control and limited transparency. To address these challenges, in this paper, we present the systematic design and evaluation of an interactive ERS, in which we introduce different levels of user control. Concretely, we introduce user control around the input (i.e., user profile), process (i.e., recommendation algorithm), and output (i.e., recommendations) of the ERS. To evaluate our system, we conducted an online user study (N=30) to explore the impact of user control on users' perceptions of the ERS in terms of several important user-centric aspects. Moreover, we investigated the effects of user control on multiple recommendation goals, namely transparency, trust, and satisfaction, as well as the interactions between these goals. Our results demonstrate the positive impact of user control on user perceived benefits of the ERS. Moreover, our study shows that user control strongly correlates with transparency and moderately correlates with trust and satisfaction. In terms of interaction between these goals, our results reveal that transparency moderately correlates and trust strongly correlates with satisfaction. Whereas, transparency and trust stand out as less correlated with each other.
Paper Structure (17 sections, 6 figures, 2 tables)

This paper contains 17 sections, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Educational Recommender Systems
  4. Interactive Recommender Systems
  5. System Design
  6. User Interface Design
  7. Interaction with the Recommendation Input
  8. Interaction with the Recommendation Process
  9. Interaction with the Recommendation Output
  10. User Evaluation
  11. Participants and Procedure
  12. User Tasks
  13. Results
  14. Perceived Benefits for Users
  15. Impact of User Control on Different Goals
  16. ...and 2 more sections

Figures (6)

  • Figure 1: User Interface of the ERS in CourseMapper with three levels of user control: input (A), process (B), and output (C)
  • Figure 2: Prototypes for different levels of user control in the ERS
  • Figure 3: Interaction with the input of the ERS
  • Figure 4: Interaction with the process and output of the ERS
  • Figure 5: Results of user evaluation based on ResQue Pu2011resque
  • ...and 1 more figures