Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AgentPeerTalk: Empowering Students through Agentic-AI-Driven Discernment of Bullying and Joking in Peer Interactions in Schools

Aditya Paul, Chi Lok Yu, Eva Adelina Susanto, Nicholas Wai Long Lau, Gwenyth Isobel Meadows

TL;DR

Distinguishing bullying from joking in school peer interactions and delivering confidential, real-time guidance to vulnerable students is a key challenge. The study evaluates an agentic AI concept by manually supplying external information to three commercial LLMs (ChatGPT-4, Gemini 1.5 Pro, Claude 3 Opus) and comparing their performance through human evaluation. Results show ChatGPT-4 benefits from the agentic augmentation, while Gemini 1.5 Pro and Claude 3 Opus display mixed or failing responses, with differences reaching statistical significance ($p=0.0041$) and attributed to political overcorrection, context-window limits, and training biases. The work contributes a practical ethical–legal–cultural–personal framework for agentic AI in schools and demonstrates the potential of real-time, context-aware support to reduce bullying's negative effects. It points to avenues for improvement, including multimodality, Mixture of Experts, and constitutional AI, to strengthen reliability and alignment in educational settings.

Abstract

Addressing school bullying effectively and promptly is crucial for the mental health of students. This study examined the potential of large language models (LLMs) to empower students by discerning between bullying and joking in school peer interactions. We employed ChatGPT-4, Gemini 1.5 Pro, and Claude 3 Opus, evaluating their effectiveness through human review. Our results revealed that not all LLMs were suitable for an agentic approach, with ChatGPT-4 showing the most promise. We observed variations in LLM outputs, possibly influenced by political overcorrectness, context window limitations, and pre-existing bias in their training data. ChatGPT-4 excelled in context-specific accuracy after implementing the agentic approach, highlighting its potential to provide continuous, real-time support to vulnerable students. This study underlines the significant social impact of using agentic AI in educational settings, offering a new avenue for reducing the negative consequences of bullying and enhancing student well-being.

AgentPeerTalk: Empowering Students through Agentic-AI-Driven Discernment of Bullying and Joking in Peer Interactions in Schools

TL;DR

Distinguishing bullying from joking in school peer interactions and delivering confidential, real-time guidance to vulnerable students is a key challenge. The study evaluates an agentic AI concept by manually supplying external information to three commercial LLMs (ChatGPT-4, Gemini 1.5 Pro, Claude 3 Opus) and comparing their performance through human evaluation. Results show ChatGPT-4 benefits from the agentic augmentation, while Gemini 1.5 Pro and Claude 3 Opus display mixed or failing responses, with differences reaching statistical significance () and attributed to political overcorrection, context-window limits, and training biases. The work contributes a practical ethical–legal–cultural–personal framework for agentic AI in schools and demonstrates the potential of real-time, context-aware support to reduce bullying's negative effects. It points to avenues for improvement, including multimodality, Mixture of Experts, and constitutional AI, to strengthen reliability and alignment in educational settings.

Abstract

Addressing school bullying effectively and promptly is crucial for the mental health of students. This study examined the potential of large language models (LLMs) to empower students by discerning between bullying and joking in school peer interactions. We employed ChatGPT-4, Gemini 1.5 Pro, and Claude 3 Opus, evaluating their effectiveness through human review. Our results revealed that not all LLMs were suitable for an agentic approach, with ChatGPT-4 showing the most promise. We observed variations in LLM outputs, possibly influenced by political overcorrectness, context window limitations, and pre-existing bias in their training data. ChatGPT-4 excelled in context-specific accuracy after implementing the agentic approach, highlighting its potential to provide continuous, real-time support to vulnerable students. This study underlines the significant social impact of using agentic AI in educational settings, offering a new avenue for reducing the negative consequences of bullying and enhancing student well-being.
Paper Structure (17 sections, 2 figures, 3 tables, 1 algorithm)

This paper contains 17 sections, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Theoretical Foundation
  3. Ethical and Legal Compliance Framework
  4. Agentic AI Solutions
  5. Results
  6. Comparing Commercial LLMs
  7. Statistical Analysis
  8. Discussion
  9. Critical Insights
  10. Limitations of the Study
  11. Conclusion
  12. Literature Review
  13. Agentic AI in Knowledge Retrieval and Distillation
  14. Bias, Hallucination, and Evaluation Benchmarks
  15. Curation of Evaluation Case Studies
  16. ...and 2 more sections

Figures (2)

  • Figure 1: AgentPeerTalk agentic process
  • Figure 2: Performance difference of LLMs using the simulated agentic approach