Experience with GitHub Copilot for Developer Productivity at Zoominfo

TL;DR

The study investigates an enterprise-scale deployment of GitHub Copilot at Zoominfo, evaluating its impact on developer productivity across 400+ engineers. Using a four-phase methodology (initial assessment, trial recruitment, two-week trial, rollout) and metrics centered on acceptance rates ($33\%$ for suggestions, $20\%$ for lines) and DevSat ($72\%$), the paper shows consistent language coverage with notable time savings around $20\%$ and substantial production contributions. Language- and editor-specific analyses reveal similar acceptance rates across major languages and nuanced differences between JetBrains and VS Code. The work offers practical deployment guidance, confirms potential productivity gains, and discusses limitations such as domain-specific reasoning and security implications for enterprise AI-assisted development.

Abstract

This paper presents a comprehensive evaluation of GitHub Copilot's deployment and impact on developer productivity at Zoominfo, a leading Go-To-Market (GTM) Intelligence Platform. We describe our systematic four-phase approach to evaluating and deploying GitHub Copilot across our engineering organization, involving over 400 developers. Our analysis combines both quantitative metrics, focusing on acceptance rates of suggestions given by GitHub Copilot and qualitative feedback given by developers through developer satisfaction surveys. The results show an average acceptance rate of 33% for suggestions and 20% for lines of code, with high developer satisfaction scores of 72%. We also discuss language-specific performance variations, limitations, and lessons learned from this medium-scale enterprise deployment. Our findings contribute to the growing body of knowledge about AI-assisted software development in enterprise settings.

Figures (9)

• Figure 1: Adoption of GitHub Copilot during the first 8 months at Zoominfo. We released licenses at a controlled pace to ensure daily usage and success.
• Figure 2: Daily data from Nov. 14th to Dec. 9th in 2024. For each day, this table shows the total number of suggestions, acceptances, lines suggested, lines accepted, acceptance rate, lines acceptance rate, lines per suggestion and lines per acceptance. The rows at the bottom show the averages (Average), standard deviations (Stdev), and medians (Median) for each relevant column. The conditional formatting of two columns in tones of green and red colors are to highlight larger values (green) and smaller values (red).
• Figure 3: Daily total number of suggestions and acceptances (a) and daily total number of suggested and accepted lines (b). Both figures cover the days from Nov. 14th to Dec. 9th in 2024. The acceptance rate in each case is the ratio of the suggested unit to the accepted unit. The trend lines are for the acceptance rates, showing a slight upward trend in each case. The wavy pattern is due to weekdays (high) and weekends (low).
• Figure 4: Acceptance rates for suggestions and lines. The former is about 1.5 times larger than the latter. The trend lines for each also show slight upward trends.
• Figure 5: Data for the top dozen languages (collected on Jan. 9th, 2025. For each language, this table shows the total number of suggestions, acceptances, lines suggested, lines accepted, acceptance rate, and lines acceptance rate. The conditional formatting of two columns in tones of green and red colors are to highlight larger values (green) and smaller values (red).