Personality-Enhanced Social Recommendations in SAMI: Exploring the Role of Personality Detection in Matchmaking

TL;DR

This study investigates integrating personality detection into SAMI to enhance matchmaking in online courses. By benchmarking zero-shot GPT models against established baselines on student introduction posts, the authors demonstrate that GPT-based approaches, particularly GPT-4o-mini, can infer Big-Five traits with competitive accuracy, despite trait-distribution biases. They prototype a personality-enhanced SAMI system that treats Extroversion, Agreeableness, and Openness as match-relevant entities, applying homophily-inspired weighting to improve social recommendations while implementing synonym-based descriptors to mitigate negative labeling. The work highlights technical feasibility and ethical considerations, and outlines future directions including real-world deployment, fairness analyses, and privacy-preserving self-hosted LLM options. This approach lays groundwork for psychologically informed educational matchmaking, with potential to strengthen online learning communities if validated in practice.

Abstract

Social belonging is a vital part of learning, yet online course environments present barriers to the organic formation of social groups. SAMI (Social Agent Mediated Interactions) offers one solution by facilitating student connections, but its effectiveness may be constrained by an incomplete Theory of Mind, limiting its ability to create an effective 'mental model' of a student. One facet of this is its inability to intuit personality, which may influence the relevance of its recommendations. To explore this gap, we examine the viability of automated personality inference by proposing a personality detection model utilizing GPT's zeroshot capability to infer Big-Five personality traits from forum introduction posts, often encouraged in online courses. We benchmark its performance against established models, finding that while GPT models show promising results on this specific dataset, performance varies significantly across traits. We identify potential biases toward optimistic trait inference, particularly for traits with skewed distributions. We demonstrate a proof-of-concept integration of personality detection into SAMI's entity-based matchmaking system, focusing on three traits with established connections to positive social formation: Extroversion, Agreeableness, and Openness. This work represents an initial exploration of personality-informed social recommendations in educational settings. While our implementation shows technical feasibility, significant questions remain. We discuss these limitations and outline directions for future work, examining what LLMs specifically capture when performing personality inference and whether personality-based matching meaningfully improves student connections in practice.