Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Great Power Brings Great Responsibility: Personalizing Conversational AI for Diverse Problem-Solvers

Italo Santos, Katia Romero Felizardo, Igor Steinmacher, Marco A. Gerosa

TL;DR

The paper addresses the onboarding barriers faced by newcomers to Open Source Software (OSS) and the bias risks inherent in large language model (LLM) guidance that may privilege certain problem-solving styles. It proposes a persona-based prompting approach, leveraging the GenderMag Abi and Tim personas, to tailor AI responses in a way that supports diverse problem solvers within OSS environments, and demonstrates the concept with illustrative prompts and comparative responses. The authors outline concrete research opportunities, including empirical studies on tailoring responses to problem-solving styles, GenderMag-based persona adaptation, and inferring personas from user interactions for real-time adaptation, supported by a replication package. The work highlights the potential for bias-mitigated, style-aware AI assistance to lower entry barriers and foster more inclusive OSS participation, while calling for further systematic evaluation and the development of best practices for responsible personalization.

Abstract

Newcomers onboarding to Open Source Software (OSS) projects face many challenges. Large Language Models (LLMs), like ChatGPT, have emerged as potential resources for answering questions and providing guidance, with many developers now turning to ChatGPT over traditional Q&A sites like Stack Overflow. Nonetheless, LLMs may carry biases in presenting information, which can be especially impactful for newcomers whose problem-solving styles may not be broadly represented. This raises important questions about the accessibility of AI-driven support for newcomers to OSS projects. This vision paper outlines the potential of adapting AI responses to various problem-solving styles to avoid privileging a particular subgroup. We discuss the potential of AI persona-based prompt engineering as a strategy for interacting with AI. This study invites further research to refine AI-based tools to better support contributions to OSS projects.

Great Power Brings Great Responsibility: Personalizing Conversational AI for Diverse Problem-Solvers

TL;DR

The paper addresses the onboarding barriers faced by newcomers to Open Source Software (OSS) and the bias risks inherent in large language model (LLM) guidance that may privilege certain problem-solving styles. It proposes a persona-based prompting approach, leveraging the GenderMag Abi and Tim personas, to tailor AI responses in a way that supports diverse problem solvers within OSS environments, and demonstrates the concept with illustrative prompts and comparative responses. The authors outline concrete research opportunities, including empirical studies on tailoring responses to problem-solving styles, GenderMag-based persona adaptation, and inferring personas from user interactions for real-time adaptation, supported by a replication package. The work highlights the potential for bias-mitigated, style-aware AI assistance to lower entry barriers and foster more inclusive OSS participation, while calling for further systematic evaluation and the development of best practices for responsible personalization.

Abstract

Newcomers onboarding to Open Source Software (OSS) projects face many challenges. Large Language Models (LLMs), like ChatGPT, have emerged as potential resources for answering questions and providing guidance, with many developers now turning to ChatGPT over traditional Q&A sites like Stack Overflow. Nonetheless, LLMs may carry biases in presenting information, which can be especially impactful for newcomers whose problem-solving styles may not be broadly represented. This raises important questions about the accessibility of AI-driven support for newcomers to OSS projects. This vision paper outlines the potential of adapting AI responses to various problem-solving styles to avoid privileging a particular subgroup. We discuss the potential of AI persona-based prompt engineering as a strategy for interacting with AI. This study invites further research to refine AI-based tools to better support contributions to OSS projects.

Paper Structure

This paper contains 5 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Related work
  3. Leveraging LLMs to Support Newcomers Problem-Solving Styles in OSS
  4. Research opportunities
  5. Conclusion

Figures (1)

  • Figure 1: Using prompt engineering in a ChatGPT by asking, "How can I submit a pull request?" The conversation was generated using GPT-4. Panel (1) shows the ChatGPT response without persona-based prompt engineering, while Panels (2) and (3) display responses tailored to the GenderMag Abi and Tim personas, respectively.