What Needs Attention? Prioritizing Drivers of Developers' Trust and Adoption of Generative AI

TL;DR

The paper tackles miscalibrated trust and adoption barriers for generative AI in software development, foregrounding cognitive diversity as a design constraint. It adopts a concurrent embedded mixed-methods approach, aggregating a large-scale survey ($N=238$) from GitHub and Microsoft, psychometrically validating a PICSE-based trust model, and applying PLS-SEM to link trust and cognitive styles to adoption, complemented by IPMA and qualitative analysis to identify high-impact design gaps. Key contributions include a validated, four-factor PICSE instrument (System/Output Quality, Functional Value, Ease of Use, Goal Maintenance), a robust model explaining substantial variance in trust ($R^2=0.68$) and adoption intentions ($R^2=0.66$), and an IPMA-informed, design-focused roadmap highlighting underperforming yet critical areas (notably goal maintenance, output accuracy, and presentation) for improving trust and inclusion. The work provides a practical guide for building trustworthy, inclusive genAI tools for developers, with implications for HAI research through a validated measurement instrument and a theory-driven prioritization of design improvements.

Abstract

Generative AI (genAI) tools promise productivity gains, yet miscalibrated trust and usage friction still hinder adoption. Moreover, genAI can be exclusionary, failing to adequately support diverse users. One such aspect of diversity is cognitive diversity, which leads to diverging interaction styles (e.g., a risk-averse developer may gate genAI outputs behind tests/review; a risk-tolerant one may prototype directly/fix issues post-hoc). When an individual's cognitive styles are unsupported, it creates additional usability barriers. Thus, to design tools that developers trust and use, we must first understand which factors shape their trust and intentions to use genAI at work? We developed a theoretical model of developers' trust and adoption of genAI through a large-scale survey (N = 238) conducted at GitHub and Microsoft. Using Partial Least Squares-Structural Equation Modeling (PLS-SEM), we found aspects related to genAI's system/output quality (e.g., presentation, safety/security, performance), functional value (e.g., educational/practical benefits), and goal maintenance (ability to sustain alignment with task goals) significantly influence trust, which, alongside developers' cognitive styles (i.e., risk tolerance, technophilic motivations, computer self-efficacy), affect adoption. An Importance-Performance Matrix Analysis (IPMA) identified high-importance factors where genAI underperforms, revealing targets for design improvement. We bolster these findings by qualitatively analyzing developers' reported challenges and risks of genAI use to uncover why these gaps persist in development contexts. We offer practical guidance for designing genAI tools that support effective, trustworthy, and inclusive developer-AI interactions.

Figures (7)

• Figure 1: Research design overview: We follow a concurrent embedded mixed-methods strategy creswell2016qualitative comprising 7 steps: ① survey design and data collection (Sec. \ref{['survey-design']}); (②–④) psychometric assessment, model development, and PLS-SEM evaluation addressing RQs1&2 (Sec. \ref{['rq1-2method']}); (⑤–⑦) IPMA, qualitative analysis, and triangulation addressing RQ3 (Sec. \ref{['RQ3-sec']}). We outline the process in Sec. \ref{['method-overview']}; details for each RQ appear in respective sections.
• Figure 2: Proposed theoretical model and measurement specification. Latent constructs (circles) are reflectively measured by survey items (squares) russo2021pls. Directed arrows between constructs indicate hypothesized relations (H1–H11); arrows from constructs to items indicate reflective measures (i.e., which items reflect a construct). Reverse-coded items are suffixed with “–R” (e.g., SE2–R). The complete questionnaire is in supplemental.
• Figure 3: PLS-SEM model results: Solid lines between constructs denote statistically significant path coefficients (p $<$ 0.05); lines to indicators show factor loadings. Dashed lines represent non-significant paths. Latent constructs are depicted as circles, and adjusted $R^2$ (Adj. $R^2$) values are reported for endogenous constructs.
• Figure 4: Importance-Performance Map Interpretation (Illustrative Example). (a) Example PLS path model, where predictors X1–X5 influence construct Y. (b) Corresponding Importance-Performance Map of Y, divided into four quadrants (Q1–Q4) based on average importance (vertical reference line) and average performance (horizontal reference line) scores. Quadrant labels are from martilla1977importance.
• Figure 5: Importance–Performance Map Analysis (IPMA) of constructs predicting trust in genAI.