Modular AI-Powered Interviewer with Dynamic Question Generation and Expertise Profiling
Aisvarya Adeseye, Jouni Isoaho, Seppo Virtanen, Mohammad Tahir
TL;DR
The paper tackles the shortcomings of fixed-script chatbots and rigid automated interviews in qualitative research by proposing an adaptive, context-aware interviewing framework. It introduces a modular, prompt-driven AI interviewer built on a locally hosted LLM, featuring five interacting modules that handle system prompting, initial question generation, expertise profiling, iterative questioning, and question-uniqueness checks, all while preserving data privacy. The approach employs a four-level expertise rubric to tailor questions and uses a local deployment of LLaMA-3.2 3B with prompts publicly accessible, demonstrating scalability and transparency across languages and cultures. Evaluation with 41 diverse participants shows high engagement and satisfaction, supporting the method's potential for privacy-preserving, scalable qualitative data collection and meaningful real-time insights. The work contributes a practical, adaptable framework and evidence for deploying AI-assisted interviews in research, education, and organizational settings, with future directions including longer context memory and multilingual expansion.
Abstract
Automated interviewers and chatbots are common in research, recruitment, customer service, and education. Many existing systems use fixed question lists, strict rules, and limited personalization, leading to repeated conversations that cause low engagement. Therefore, these tools are not effective for complex qualitative research, which requires flexibility, context awareness, and ethical sensitivity. Consequently, there is a need for a more adaptive and context-aware interviewing system. To address this, an AI-powered interviewer that dynamically generates questions that are contextually appropriate and expertise aligned is presented in this study. The interviewer is built on a locally hosted large language model (LLM) that generates coherent dialogue while preserving data privacy. The interviewer profiles the participants' expertise in real time to generate knowledge-appropriate questions, well-articulated responses, and smooth transition messages similar to human-like interviews. To implement these functionalities, a modular prompt engineering pipeline was designed to ensure that the interview conversation remains scalable, adaptive, and semantically rich. To evaluate the AI-powered interviewer, it was tested with various participants, and it achieved high satisfaction (mean 4.45) and engagement (mean 4.33). The proposed interviewer is a scalable, privacy-conscious solution that advances AI-assisted qualitative data collection.