Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews

Hasindri Watawana, Sergio Burdisso, Diego A. Moreno-Galván, Fernando Sánchez-Vega, A. Pastor López-Monroy, Petr Motlicek, Esaú Villatoro-Tello

Abstract

Automatic depression detection from doctor-patient conversations has gained momentum thanks to the availability of public corpora and advances in language modeling. However, interpretability remains limited: strong performance is often reported without revealing what drives predictions. We analyze three datasets: ANDROIDS, DAIC-WOZ, E-DAIC and identify a systematic bias from interviewer prompts in semi-structured interviews. Models trained on interviewer turns exploit fixed prompts and positions to distinguish depressed from control subjects, often achieving high classification scores without using participant language. Restricting models to participant utterances distributes decision evidence more broadly and reflects genuine linguistic cues. While semi-structured protocols ensure consistency, including interviewer prompts inflates performance by leveraging script artifacts. Our results highlight a cross-dataset, architecture-agnostic bias and emphasize the need for analyses that localize decision evidence by time and speaker to ensure models learn from participants' language.

When Consistency Becomes Bias: Interviewer Effects in Semi-Structured Clinical Interviews

Abstract

Automatic depression detection from doctor-patient conversations has gained momentum thanks to the availability of public corpora and advances in language modeling. However, interpretability remains limited: strong performance is often reported without revealing what drives predictions. We analyze three datasets: ANDROIDS, DAIC-WOZ, E-DAIC and identify a systematic bias from interviewer prompts in semi-structured interviews. Models trained on interviewer turns exploit fixed prompts and positions to distinguish depressed from control subjects, often achieving high classification scores without using participant language. Restricting models to participant utterances distributes decision evidence more broadly and reflects genuine linguistic cues. While semi-structured protocols ensure consistency, including interviewer prompts inflates performance by leveraging script artifacts. Our results highlight a cross-dataset, architecture-agnostic bias and emphasize the need for analyses that localize decision evidence by time and speaker to ensure models learn from participants' language.

Paper Structure

This paper contains 14 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Datasets
  3. DAIC-WOZ
  4. E-DAIC
  5. ANDROIDS
  6. Methodology
  7. Experiments and Results
  8. Analysis and Discussion
  9. Conclusion
  10. Acknowledgments
  11. Ethics Statement
  12. Limitations
  13. Bibliographical References
  14. Language Resource References

Figures (2)

  • Figure 1: Temporal heatmaps comparing keyword evidence learned by interviewer-only (I, top) vs. participant-only (P, bottom) models across interviews in the ANDROIDS and E-DAIC datasets. Each column represents one interview. The y-axis corresponds to the normalized interview timeline, where 0% marks the beginning of the interview and 100% marks its end. White vertical lines denote split boundaries (train/dev/test for E-DAIC; train/dev only for ANDROIDS). The ANDROIDS plot is shown for Fold 1.
  • Figure 2: Color-coded interview excerpts in which prompts identified by the I-model as bias-carrying are highlighted. Underlined words denote the model’s learned keywords, corresponding to the high-contrast narrow bands in Figure \ref{['fig:heatmaps-side-by-side']}.