Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Human-Centered Development of Indicators for Self-Service Learning Analytics: A Transparency through Exploration Approach

Shoeb Joarder, Mohamed Amine Chatti

TL;DR

This paper tackles transparency and trust in learning analytics by introducing the Indicator Editor, a self-service, no-code/low-code tool enabling stakeholders with data skills to implement LA indicators through a transparency-through-exploration approach. Built on human-centered design, the Indicator Editor guides users through an iterative, end-to-end indicator development process, including datasets, filters, analysis, and visualizations, and outputs an embeddable Interactive Indicator Code ($IIC$). A qualitative study with $n=15$ participants provides initial evidence that granting end-users control over indicator implementation positively affects perceived transparency, trust, satisfaction, and acceptance of LA systems, while also highlighting cognitive-load considerations and the need for adaptable interfaces. The work contributes design guidance for SSLA in educational settings and outlines future work to broaden accessibility, extend indicator complexity, and validate effects with quantitative methods.

Abstract

The aim of learning analytics is to turn educational data into insights, decisions, and actions to improve learning and teaching. The reasoning of the provided insights, decisions, and actions is often not transparent to the end-user, and this can lead to trust and acceptance issues when interventions, feedback, and recommendations fail. In this paper, we shed light on achieving transparent learning analytics by following a transparency through exploration approach. To this end, we present the design, implementation, and evaluation details of the Indicator Editor, which aims to support self-service learning analytics (SSLA) by empowering end-users to take control of the indicator implementation process. We systematically designed and implemented the Indicator Editor through an iterative human-centered design (HCD) approach. Further, we conducted a qualitative user study (n=15) to investigate the impact of following an SSLA approach on the users' perception of and interaction with the Indicator Editor. Our study showed qualitative evidence that supporting user interaction and providing user control in the indicator implementation process can have positive effects on different crucial aspects of learning analytics, namely transparency, trust, satisfaction, and acceptance.

Human-Centered Development of Indicators for Self-Service Learning Analytics: A Transparency through Exploration Approach

TL;DR

This paper tackles transparency and trust in learning analytics by introducing the Indicator Editor, a self-service, no-code/low-code tool enabling stakeholders with data skills to implement LA indicators through a transparency-through-exploration approach. Built on human-centered design, the Indicator Editor guides users through an iterative, end-to-end indicator development process, including datasets, filters, analysis, and visualizations, and outputs an embeddable Interactive Indicator Code (). A qualitative study with participants provides initial evidence that granting end-users control over indicator implementation positively affects perceived transparency, trust, satisfaction, and acceptance of LA systems, while also highlighting cognitive-load considerations and the need for adaptable interfaces. The work contributes design guidance for SSLA in educational settings and outlines future work to broaden accessibility, extend indicator complexity, and validate effects with quantitative methods.

Abstract

The aim of learning analytics is to turn educational data into insights, decisions, and actions to improve learning and teaching. The reasoning of the provided insights, decisions, and actions is often not transparent to the end-user, and this can lead to trust and acceptance issues when interventions, feedback, and recommendations fail. In this paper, we shed light on achieving transparent learning analytics by following a transparency through exploration approach. To this end, we present the design, implementation, and evaluation details of the Indicator Editor, which aims to support self-service learning analytics (SSLA) by empowering end-users to take control of the indicator implementation process. We systematically designed and implemented the Indicator Editor through an iterative human-centered design (HCD) approach. Further, we conducted a qualitative user study (n=15) to investigate the impact of following an SSLA approach on the users' perception of and interaction with the Indicator Editor. Our study showed qualitative evidence that supporting user interaction and providing user control in the indicator implementation process can have positive effects on different crucial aspects of learning analytics, namely transparency, trust, satisfaction, and acceptance.

Paper Structure

This paper contains 28 sections, 8 figures.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Transparent Learning Analytics
  4. Human-Centered Learning Analytics
  5. Indicator Editor
  6. User Scenarios
  7. Teacher Scenario:
  8. Student Scenario:
  9. Design Goals
  10. Iterative Design
  11. First iteration, initial design, low-fidelity prototypes:
  12. Second iteration, high-fidelity prototypes
  13. Final prototypes and implementation
  14. Datasets:
  15. Filters:
  16. ...and 13 more sections

Figures (8)

  • Figure 1: User scenario - Teachers/students use Indicator Editor to create indicators for their CourseMapper dashboard
  • Figure 2: Paper Prototypes of the Indicator Editor
  • Figure 3: High-fidelity Prototypes of the Indicator Editor: Dataset and Filters
  • Figure 4: High-fidelity Prototypes of the Indicator Editor: Analysis, Visualization, and Dashboard
  • Figure 5: Final Prototypes of the Indicator Editor
  • ...and 3 more figures