Coaching Copilot: Blended Form of an LLM-Powered Chatbot and a Human Coach to Effectively Support Self-Reflection for Leadership Growth

TL;DR

This work addresses the challenge of fostering deep self-reflection for leadership growth, proposing a blended approach that combines an LLM-powered chatbot with human coaches. Through a design workshop and a two-week field study with ten coach–client pairs, it provides empirical evidence on where chatbots can help, what roles humans should play, and how clients perceive and engage with the technology. The authors offer an actionable guideline for deploying chatbot-assisted reflection in executive coaching and delineate the current limits of chatbot capability, emphasizing the value of human-in-the-loop for deeper introspection. The findings have practical implications for scalable leadership development and suggest broader applicability of blended AI–human coaching in other reflection-intensive settings.

Abstract

Chatbots' role in fostering self-reflection is now widely recognized, especially in inducing users' behavior change. While the benefits of 24/7 availability, scalability, and consistent responses have been demonstrated in contexts such as healthcare and tutoring to help one form a new habit, their utilization in coaching necessitating deeper introspective dialogue to induce leadership growth remains unexplored. This paper explores the potential of such a chatbot powered by recent Large Language Models (LLMs) in collaboration with professional coaches in the field of executive coaching. Through a design workshop with them and two weeks of user study involving ten coach-client pairs, we explored the feasibility and nuances of integrating chatbots to complement human coaches. Our findings highlight the benefits of chatbots' ubiquity and reasoning capabilities enabled by LLMs while identifying their limitations and design necessities for effective collaboration between human coaches and chatbots. By doing so, this work contributes to the foundation for augmenting one's self-reflective process with prevalent conversational agents through the human-in-the-loop approach.

Figures (8)

• Figure 1: The prototype interface to use an LLM-powered chatbot coach. In the user study, this prototype was provided to clients and used at their own pace for two weeks to support their reflection to accomplish their professional goals. The presented conversation was derived from the use of a prototype by one of the authors to demonstrate its behavior.
• Figure 2: The prompt used in the chatbot guided by the workshop with coaches.
• Figure 3: The procedure of the user study using the developed chatbot text coach as a supplement to the regular coaching.
• Figure 4: (Left) The number of messages each client sent to the chatbot coach per session. (Right) The total length of the messages per session in the number of characters. The blue area highlights indicate the 95% confidence interval of the average value.
• Figure 5: Example from one client's messages with the chatbot coach. It is observed that the chatbot often acknowledges the client's actions and asks questions that can further break down the problems they face.