Examining the Role of Peer Acknowledgements on Social Annotations: Unraveling the Psychological Underpinnings

TL;DR

This study investigates how peer acknowledgement, operationalized via upvotes, influences learners' engagement in digital social annotations and examines the linguistic signals of underlying psychology in annotation text. Using Perusall log data from 91 undergraduates, cross-lag regression establishes that receiving peer acknowledgement at time $t-1$ significantly increases future annotation Initiation ($\beta = 0.187$, $p < .001$) and responses ($\beta = 0.052$, $p = .009$). Text mining with LIWC across 20 features and Shapley-value analysis via Gradient Boosting Revealer identifies positive indicators in affect, cognition, and motivation (e.g., $curiosity$, $emo\_pos$, $insight$, $tentat$, $achievement$) as predictors of peer acknowledgement, while certain negative or social cues diminish it; GBR explains approximately $R^2 \approx 0.92$ on both cross-validation and test data. The findings offer actionable guidance for educators and platform designers to cultivate engaging online communities by emphasizing positive affect, open discussion, and motivated cognitive engagement, and by leveraging learning analytics to prompt reflective and constructive interactions.

Abstract

This study explores the impact of peer acknowledgement on learner engagement and implicit psychological attributes in written annotations on an online social reading platform. Participants included 91 undergraduates from a large North American University. Using log file data, we analyzed the relationship between learners' received peer acknowledgement and their subsequent annotation behaviours using cross-lag regression. Higher peer acknowledgements correlate with increased initiation of annotations and responses to peer annotations. By applying text mining techniques and calculating Shapley values to analyze 1,969 social annotation entries, we identified prominent psychological themes within three dimensions (i.e., affect, cognition, and motivation) that foster peer acknowledgment in digital social annotation. These themes include positive affect, openness to learning and discussion, and expression of motivation. The findings assist educators in improving online learning communities and provide guidance to technology developers in designing effective prompts, drawing from both implicit psychological cues and explicit learning behaviours.

Figures (3)

• Figure 1: The Interface of the Social Annotation Platform Perusall.
• Figure 2: Summary of the Shapley values across all 20 linguistic features, categorized into four dimensions. Each value signifies the average influence of its respective feature on the model’s output, illustrating not only the magnitude but also the direction of the impact.
• Figure 3: Fine-grained visualization of Shapley values with respect to those selected linguistic features across all possible feature combinations. The left hand side shows the four features that are strong positive indicators, while the right hand side shows the four features that are strong negative indicators. Points of red denote higher feature values, while points of blue represent lower values.