Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OnDiscuss: An Epistemic Network Analysis Learning Analytics Visualization Tool for Evaluating Asynchronous Online Discussions

Yanye Luther, Marcia Moraes, Sudipto Ghosh, James Folkestad

TL;DR

The paper addresses the challenge of efficiently evaluating asynchronous online discussions by introducing OnDiscuss, a learning analytics visualization tool that fuses text mining with Epistemic Network Analysis (ENA). OnDiscuss automatically generates a five-topic, ten-keyword initial codebook via Latent Dirichlet Allocation and updates ENA networks in real time as instructors edit the codebook, with Canvas integration for easy access. Through an empirical observational study involving a novice and an expert ENA user, the work assesses usability, interpretability, and potential reductions in grading effort, finding that novices can learn to interpret ENA visuals with guidance while experts benefit from baseline comparisons and modular networks. The findings suggest that OnDiscuss can broaden ENA adoption by abstracting technical details and enabling instructor-driven customization, though limitations such as reliance on text data and small, biased samples warrant further validation across disciplines and larger class sizes.

Abstract

Asynchronous online discussions are common assignments in both hybrid and online courses to promote critical thinking and collaboration among students. However, the evaluation of these assignments can require considerable time and effort from instructors. We created OnDiscuss, a learning analytics visualization tool for instructors that utilizes text mining algorithms and Epistemic Network Analysis (ENA) to generate visualizations of student discussion data. Text mining is used to generate an initial codebook for the instructor as well as automatically code the data. This tool allows instructors to edit their codebook and then dynamically view the resulting ENA networks for the entire class and individual students. Through empirical investigation, we assess this tool's effectiveness to help instructors in analyzing asynchronous online discussion assignments.

OnDiscuss: An Epistemic Network Analysis Learning Analytics Visualization Tool for Evaluating Asynchronous Online Discussions

TL;DR

The paper addresses the challenge of efficiently evaluating asynchronous online discussions by introducing OnDiscuss, a learning analytics visualization tool that fuses text mining with Epistemic Network Analysis (ENA). OnDiscuss automatically generates a five-topic, ten-keyword initial codebook via Latent Dirichlet Allocation and updates ENA networks in real time as instructors edit the codebook, with Canvas integration for easy access. Through an empirical observational study involving a novice and an expert ENA user, the work assesses usability, interpretability, and potential reductions in grading effort, finding that novices can learn to interpret ENA visuals with guidance while experts benefit from baseline comparisons and modular networks. The findings suggest that OnDiscuss can broaden ENA adoption by abstracting technical details and enabling instructor-driven customization, though limitations such as reliance on text data and small, biased samples warrant further validation across disciplines and larger class sizes.

Abstract

Asynchronous online discussions are common assignments in both hybrid and online courses to promote critical thinking and collaboration among students. However, the evaluation of these assignments can require considerable time and effort from instructors. We created OnDiscuss, a learning analytics visualization tool for instructors that utilizes text mining algorithms and Epistemic Network Analysis (ENA) to generate visualizations of student discussion data. Text mining is used to generate an initial codebook for the instructor as well as automatically code the data. This tool allows instructors to edit their codebook and then dynamically view the resulting ENA networks for the entire class and individual students. Through empirical investigation, we assess this tool's effectiveness to help instructors in analyzing asynchronous online discussion assignments.
Paper Structure (14 sections, 5 figures, 3 tables)

This paper contains 14 sections, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Participant Selection
  5. OnDiscuss Description and Functionality
  6. Experimental Procedure
  7. Results
  8. Instructor A
  9. Instructor B
  10. Discussion
  11. RQ1. Are ENA visualizations of asynchronous online discussion data helpful for instructors who are novices in ENA?
  12. RQ2. Do the ENA visualizations have the potential to reduce the time and effort spent assessing asynchronous online discussions for instructors of any experience level with ENA?
  13. Limitations
  14. Conclusion and Future Works

Figures (5)

  • Figure 1: Example Class/Group ENA View for a Discussion Topic
  • Figure 2: Example Individual ENA View for a Discussion Topic
  • Figure 3: Instructor A Individual ENA Visualizations
  • Figure 4: Instructor A Group ENA Visualizations
  • Figure 5: Instructor B Group ENA Visualizations