From First Patch to Long-Term Contributor: Evaluating Onboarding Recommendations for OSS Newcomers

TL;DR

This study addresses the generalizability of OSS onboarding recommendations by combining a systematic literature review with a large-scale empirical evaluation. It identifies 15 task-related recommendations for newcomers, categorized into task selection, preparation, and completion, then tests their associations with first-patch acceptance using data from 5 Gerrit-based and 1,155 GitHub projects via multivariate logistic regression. The results reveal a spectrum of outcomes: several recommendations positively correlate with acceptance in many contexts, others are context-dependent or negatively associated, and some newcomer-specific strategies should be abandoned after initial onboarding. Overall, the work provides nuanced guidance for adaptive onboarding practices and highlights the importance of considering project platform, contributor status, and task characteristics when advising newcomers.

Abstract

Attracting and retaining a steady stream of new contributors is crucial to ensuring the long-term survival of open-source software (OSS) projects. However, there are two key research gaps regarding recommendations for onboarding new contributors to OSS projects. First, most of the existing recommendations are based on a limited number of projects, which raises concerns about their generalizability. If a recommendation yields conflicting results in a different context, it could hinder a newcomer's onboarding process rather than help them. Second, it's unclear whether these recommendations also apply to experienced contributors. If certain recommendations are specific to newcomers, continuing to follow them after their initial contributions are accepted could hinder their chances of becoming long-term contributors. To address these gaps, we conducted a two-stage mixed-method study. In the first stage, we conducted a Systematic Literature Review (SLR) and identified 15 task-related actionable recommendations that newcomers to OSS projects can follow to improve their odds of successful onboarding. In the second stage, we conduct a large-scale empirical study of five Gerrit-based projects and 1,155 OSS projects from GitHub to assess whether those recommendations assist newcomers' successful onboarding. Our results suggest that four recommendations positively correlate with newcomers' first patch acceptance in most contexts. Four recommendations are context-dependent, and four indicate significant negative associations for most projects. Our results also found three newcomer-specific recommendations, which OSS joiners should abandon at non-newcomer status to increase their odds of becoming long-term contributors.

Figures (1)

• Figure 1: An overview of our two-stage mixed method research design