Characterizing Faculty Online Learning Community Interactions Using Social Network Analysis

TL;DR

This work employs Social Network Analysis to characterize how PICUP members interact on Slack and to assess alignment with goals of lowering barriers to computation in physics, sustaining community growth, and developing leadership. By converting Slack messages from eight channels into directed, weighted networks and comparing observed metrics to null ensembles generated by the Configuration Model, the study identifies three engagement levels (Active, Passive, Receivers) and reveals generally low reciprocity and limited broad participation across channels. The findings suggest that, in its current Slack form, PICUP does not fully realize its online-governance goals, though high WGCC values in some channels indicate localized clustering among active participants. The paper highlights the value of a multimethod approach, combining SNA with qualitative and content-based analyses, to obtain a more complete picture of community health and guide future improvements for online faculty communities.

Abstract

The Partnership for Integration of Computation into Undergraduate Physics (PICUP) was founded in the mid-2010s to assist educators with the challenges of integrating computation into physics curricula. In addition to in-person trainings and hosted educational materials, PICUP uses a Slack Workspace to continue collaboration and discussion offline. In this work, we use Social Network Analysis (SNA) to study the communication patterns of PICUP and assess if PICUP is meeting their goals in the Slack environment. Through our analysis, we discuss PICUP's community structure and define a conceptual framework to evaluate if the goals are being met through SNA metrics. We present a comprehensive analysis of eight channels in the Slack Workspace using various SNA metrics, identifying three distinct levels of user engagement. We conclude with implications for PICUP and provide recommendations for the community.

Figures (18)

• Figure 1: A snapshot of the restructured data from the Glowscript channel. The figure displays the post-processed data following our steps to identify receivers and senders, transform messages sent to the entire channel, and represent emoji responses as messages.
• Figure 2: We present an example to show how the structured data is represented as a network in our project. Users are represented with nodes, messages are represented with directed edges, using size and color to indicate user attributes. In our representation, solid edges indicate direct messages between two users. Opposingly, dashed edges indicate a message that was sent from one user to the entire channel. Edge width corresponds to the number of messages between two users.
• Figure 3: Differentiating transitive and intransitive triplets. Triplets defined by wasserman1994social are a 2-path - there is a path from node A to B and node B to C as seen in (2). A transitive triplet follows the transitive property implying that if there is a triplet from nodes A to B and B to C, the triplet is transitive if there exists a path from A to C (as seen in (3))opsahl2009clustering.
• Figure 4: For the weighted and directed GCC, we calculate a triplet value for each counted triplet in the metric. We use the geometric mean as shown. The geometric mean for an open and closed triplet with edge weights of $3$ and $4$ is $\sqrt{4*3}$
• Figure 5: The strength distributions for the main networks analyzed in Section \ref{['sec:SNA']} - Figure \ref{['fig:AT_strength']} for the Advanced Thermodynamics, Figure \ref{['fig:J_strength']} for the Jupyter, Figure \ref{['fig:CP_strength']} for the Classroom Pedagogy.