To Search or To Gen? Exploring the Synergy between Generative AI and Web Search in Programming

TL;DR

This paper addresses how programmers can effectively combine web search and generative AI for programming tasks, challenging the view that these tools are mere substitutes. Through eight in-depth interviews, it identifies three decision-making stages (selection, extraction, translation) and three corresponding challenges, then proposes a unified process model that integrates the information-foraging loop with sensemaking. The key contributions are the empirical grounding of tool-interaction factors, a structured three-stage framework, and a holistic model to guide future tool design for synergistic use. The findings have practical impact for developing integrated IDEs and user workflows that better leverage both web search and generative AI in real-world programming tasks.

Abstract

The convergence of generative AI and web search is reshaping problem-solving for programmers. However, the lack of understanding regarding their interplay in the information-seeking process often leads programmers to perceive them as alternatives rather than complementary tools. To analyze this interaction and explore their synergy, we conducted an interview study with eight experienced programmers. Drawing from the results and literature, we have identified three major challenges and proposed three decision-making stages, each with its own relevant factors. Additionally, we present a comprehensive process model that captures programmers' interaction patterns. This model encompasses decision-making stages, the information-foraging loop, and cognitive activities during system interaction, offering a holistic framework to comprehend and optimize the use of these convergent tools in programming.

Figures (2)

• Figure 1: Three decision-making stages with key factors influencing these decisions and the challenges associated with them.
• Figure 2: A process model describing interactions between programmers and search/generative AI.