Conversate: Supporting Reflective Learning in Interview Practice Through Interactive Simulation and Dialogic Feedback

TL;DR

Conversate is a web-based application that supports reflective learning in job interview practice by leveraging large language models (LLMs) for interactive interview simulations and dialogic feedback and participants found the LLM-supported dialogic feedback to be beneficial.

Abstract

Job interviews play a critical role in shaping one's career, yet practicing interview skills can be challenging, especially without access to human coaches or peers for feedback. Recent advancements in large language models (LLMs) present an opportunity to enhance the interview practice experience. Yet, little research has explored the effectiveness and user perceptions of such systems or the benefits and challenges of using LLMs for interview practice. Furthermore, while prior work and recent commercial tools have demonstrated the potential of AI to assist with interview practice, they often deliver one-way feedback, where users only receive information about their performance. By contrast, dialogic feedback, a concept developed in learning sciences, is a two-way interaction feedback process that allows users to further engage with and learn from the provided feedback through interactive dialogue. This paper introduces Conversate, a web-based application that supports reflective learning in job interview practice by leveraging large language models (LLMs) for interactive interview simulations and dialogic feedback. To start the interview session, the user provides the title of a job position (e.g., entry-level software engineer) in the system. Then, our system will initialize the LLM agent to start the interview simulation by asking the user an opening interview question and following up with questions carefully adapted to subsequent user responses. After the interview session, our back-end LLM framework will then analyze the user's responses and highlight areas for improvement. Users can then annotate the transcript by selecting specific sections and writing self-reflections. Finally, the user can interact with the system for dialogic feedback, conversing with the LLM agent to learn from and iteratively refine their answers based on the agent's guidance.

Figures (5)

• Figure 1: Step 1: Interview Simulation. The AI agent conducts an interactive interview simulation, asking initial questions (e.g., "Tell me about yourself?") and dynamically generating contextually relevant follow-up questions based on the user’s responses using an LLM. Note: The pixelated area shows the participants who appeared on camera. It has been pixelated to comply with anonymization rules.
• Figure 2: Step 2: AI-Assisted Annotation. After the simulation, the system analyzes the user’s responses and highlights areas for improvement (A). Users can then select specific transcript sections (B, C), either based on their own preferences or the highlighted areas, and provide self-reflections (D).
• Figure 3: Step 3: Dialogic Feedback. The user engages in dialogic feedback focused on annotated moments, learning from and iteratively refining their understanding based on the AI agent's guidance. (1) The user initiates the interaction by asking a question (e.g., "How can I improve this part?"), followed by AI feedback. (2) The user refines their answer based on the feedback by clicking the microphone button to record their revised answer verbally. (3) The user receives further feedback or affirmation from the AI. This iterative process of refining and receiving feedback can continue if the user's answer is not satisfactory yet or if the user wants to improve their answer further.
• Figure 4: Sample Conversation During Interview Practice (P9): The text highlighted in orange indicates repeated keywords that made participants feel the follow-up questions were contextually relevant. These follow-up questions encouraged participants to delve deeper into their responses.
• Figure 5: Expressing Disagreement in Dialogic Feedback (P6)