Deploying Chatbots in Customer Service: Adoption Hurdles and Simple Remedies
Evgeny Kagan, Brett Hathaway, Maqbool Dada
TL;DR
This paper investigates why AI-powered chatbots underperform in customer service and how design choices influence adoption. It combines a retrospective survey, a stylized gatekeeper model, and three incentivized experiments to identify adoption hurdles—chiefly gatekeeper aversion and algorithm aversion—and test low-cost remedies such as transparency and waiting-time nudges. The authors implement a random-utility framework within an $M/D/1$ queue to quantify staffing-cost savings, finding up to $19.7 ext{ extperthousand}$ savings under moderate congestion when remedies are applied. Practical implications show that simple service-design interventions can meaningfully raise chatbot uptake and reduce costs, while methodological insights reveal that realism in measuring algorithm aversion strengthens observed effects.
Abstract
Despite recent advances in Artificial Intelligence, the use of chatbot technology in customer service continues to face adoption hurdles. This paper explores reasons for these adoption hurdles and tests several service design levers to increase chatbot uptake. We use incentivized online experiments to study chatbot uptake in a variety of scenarios. The results of these experiments are threefold. First, people respond positively to improvements in chatbot performance; however, the chatbot channel is utilized less frequently than expected-time minimization would predict. A key driver of this underutilization is the reluctance to engage with a gatekeeper process, i.e., a process with an imperfect initial service stage and possible transfer to a second, expert service stage -- a behavior we term "gatekeeper aversion". We show that gatekeeper aversion can be further amplified by a secondary hurdle, algorithm aversion. Second, chatbot uptake can be increased by providing customers with average waiting times in the chatbot channel, as well as by being more transparent about chatbot capabilities and limitations. Third, methodologically, we show that chatbot adoption can depend on experimental implementation. In particular, chatbot adoption decreases further as (i) stakes are increased, (ii) the human/algorithmic nature of the server is manipulated with more realism. Our results suggest that firms should continue to prioritize investments in chatbot technology. However, less expensive, process-related interventions can also be effective. These may include being more transparent about the types of queries that are (or are not) suitable for chatbots, emphasizing chatbot reliability and quick resolution times, as well as providing faster live agent access to customers who experienced chatbot failure.