Leveraging ChatGPT and Other NLP Methods for Identifying Risk and Protective Behaviors in MSM: Social Media and Dating apps Text Analysis

TL;DR

The paper addresses HIV risk and substance use among MSM by leveraging text data from social media and dating apps to identify risk and protective behaviors. It combines multiple NLP features, including risk-word lexicons, BERT and LIWC embeddings, and novel ChatGPT-based embeddings, with Fisher-score feature selection to predict survey-derived outcomes such as binge drinking, multiple partners, PrEP uptake, and AUDIT-C risk. The study reports F1 scores up to 0.78 for binge drinking and multiple partners, with moderate performance for PrEP uptake and heavy drinking (0.64 and 0.63); ChatGPT-based risk-word embeddings are particularly informative. The findings demonstrate feasibility of scalable, personalized public health interventions for MSM using large-language-model-based text representations, while also highlighting the need for larger datasets and further validation to translate into practice.

Abstract

Men who have sex with men (MSM) are at elevated risk for sexually transmitted infections and harmful drinking compared to heterosexual men. Text data collected from social media and dating applications may provide new opportunities for personalized public health interventions by enabling automatic identification of risk and protective behaviors. In this study, we evaluated whether text from social media and dating apps can be used to predict sexual risk behaviors, alcohol use, and pre-exposure prophylaxis (PrEP) uptake among MSM. With participant consent, we collected textual data and trained machine learning models using features derived from ChatGPT embeddings, BERT embeddings, LIWC, and a dictionary-based risk term approach. The models achieved strong performance in predicting monthly binge drinking and having more than five sexual partners, with F1 scores of 0.78, and moderate performance in predicting PrEP use and heavy drinking, with F1 scores of 0.64 and 0.63. These findings demonstrate that social media and dating app text data can provide valuable insights into risk and protective behaviors and highlight the potential of large language model-based methods to support scalable and personalized public health interventions for MSM.