Exploring Culturally Informed AI Assistants: A Comparative Study of ChatBlackGPT and ChatGPT
Lisa Egede, Ebtesam Al Haque, Gabriella Thompson, Alicia Boyd, Angela D. R. Smith, Brittany Johnson
TL;DR
This paper investigates whether culturally informed AI assistants improve responses for Black users compared with a general-purpose tool, focusing on travel-related inquiries. Using CultureBank prompts drawn from Reddit and TikTok (130 Reddit and 141 TikTok questions), the authors apply a mixed-methods approach—sentiment analysis with a RoBERTa model, readability metrics, and inductive thematic analysis—to compare ChatBlackGPT with ChatGPT. Findings indicate that ChatBlackGPT contextualizes Black history, provides concrete resources, and maintains a measured, empathetic tone, while ChatGPT offers similar structure but less culturally anchored content. The work highlights the potential of culturally tailored AI to enhance engagement for Black users and outlines follow-up studies (surveys, interviews, and workshops) to further design AI tools centered on the Black lived experience.
Abstract
In recent years, we have seen an influx in reliance on AI assistants for information seeking. Given this widespread use and the known challenges AI poses for Black users, recent efforts have emerged to identify key considerations needed to provide meaningful support. One notable effort is the development of ChatBlackGPT, a culturally informed AI assistant designed to provide culturally relevant responses. Despite the existence of ChatBlackGPT, there is no research on when and how Black communities might engage with culturally informed AI assistants and the distinctions between engagement with general purpose tools like ChatGPT. To fill this gap, we propose a research agenda grounded in results from a preliminary comparative analysis of outputs provided by ChatGPT and ChatBlackGPT for travel-related inquiries. Our efforts thus far emphasize the need to consider Black communities' values, perceptions, and experiences when designing AI assistants that acknowledge the Black lived experience.