Designing a Dashboard for Transparency and Control of Conversational AI

TL;DR

This work tackles the problem of transparency in conversational AI by exposing and enabling control over the system's internal representation of the user. It introduces TalkTuner, an end-to-end prototype that connects interpretability techniques (linear probes and activation-based control) with a user-facing dashboard to display and adjust four demographic attributes in real time. Through a design-probe methodology and a user study with 19 participants, the paper demonstrates that exposing internal states can enhance user understanding, reveal biases, and increase a sense of control, while also highlighting privacy concerns and the need for careful design. The findings offer a practical design pathway for instrumented, more transparent chatbots and propose concrete directions for expanding attributes, refining controls, and extending to other modalities.

Abstract

Conversational LLMs function as black box systems, leaving users guessing about why they see the output they do. This lack of transparency is potentially problematic, especially given concerns around bias and truthfulness. To address this issue, we present an end-to-end prototype-connecting interpretability techniques with user experience design-that seeks to make chatbots more transparent. We begin by showing evidence that a prominent open-source LLM has a "user model": examining the internal state of the system, we can extract data related to a user's age, gender, educational level, and socioeconomic status. Next, we describe the design of a dashboard that accompanies the chatbot interface, displaying this user model in real time. The dashboard can also be used to control the user model and the system's behavior. Finally, we discuss a study in which users conversed with the instrumented system. Our results suggest that users appreciate seeing internal states, which helped them expose biased behavior and increased their sense of control. Participants also made valuable suggestions that point to future directions for both design and machine learning research. The project page and video demo of our TalkTuner system are available at https://bit.ly/talktuner-project-page

Figures (23)

• Figure 1: Reading probe's validation accuracy across layers.
• Figure 2: Dashboard interface. (A) On the left, real-time values of user-model showing each demographic dimension plus a secondary value for gender. (B) The user modifies "Gender" dimension by pinning down "Male." (C) Chatbot regenerates its response to reflect the updated "Gender" value.
• Figure 3: User-model accuracy measured by chat turn in study sessions.
• Figure 4: Questionnaire responses with Wilcoxon signed rank test. See Appendix \ref{['appendix:post-task-questionnaires']} for full-length questions.
• Figure 5: Effect of training data size on the reading and control probe's performance. The accuracy is measured on a held-out validation set of each attribute. Probes were trained and validated on the internal representation at 30th layer. In the plots above, the starting training size for gender is 64, for age and socioeconomic status attribute is 80, for education is 90.