Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fine-Tuning Large Audio-Language Models with LoRA for Precise Temporal Localization of Prolonged Exposure Therapy Elements

Suhas BN, Andrew M. Sherrill, Jyoti Alaparthi, Dominik Mattioli, Rosa I. Arriaga, Chris W. Wiese, Saeed Abdullah

TL;DR

The paper tackles the bottleneck of PE therapy fidelity assessment by automatically localizing three key fidelity phases (P1, P2, P3) in session audio using a privacy-preserving, LoRA-finetuned Qwen2-Audio on 30-second windows. It introduces a soft-supervision labeling pipeline with LLM-generated timestamps and rater verification, and demonstrates that the approach achieves a mean absolute error of about 5.3 seconds on 308 sessions, competitive with human tolerance. The work analyzes the impact of window size and LoRA rank, revealing a context granularity trade-off and highlighting the practicality of automatic, fine-grained fidelity tracking for clinician training and quality assurance. It offers a scalable framework applicable to other protocol-driven conversational domains while preserving data privacy in clinical settings.

Abstract

Prolonged Exposure (PE) therapy is an effective treatment for post-traumatic stress disorder (PTSD), but evaluating therapist fidelity remains labor-intensive due to the need for manual review of session recordings. We present a method for the automatic temporal localization of key PE fidelity elements, identifying their start and stop times, directly from session audio and transcripts. Our approach fine-tunes a large pre-trained audio-language model, Qwen2-Audio, using Low-Rank Adaptation (LoRA) to process focused 30-second windows of audio-transcript input. Fidelity labels for three core protocol phases, therapist orientation (P1), imaginal exposure (P2), and post-imaginal processing (P3), are generated via LLM-based prompting and verified by trained raters. The model is trained to predict normalized boundary offsets using soft supervision guided by task-specific prompts. On a dataset of 308 real PE sessions, our best configuration (LoRA rank 8, 30s windows) achieves a mean absolute error (MAE) of 5.3s across tasks, within typical rater tolerance for timestamp review, enabling practical fidelity QC. We further analyze the effects of window size and LoRA rank, highlighting the importance of context granularity and model adaptation. This work introduces a privacy-preserving, scalable framework for fidelity tracking in PE therapy, with potential to support clinician training, supervision, and quality assurance.

Fine-Tuning Large Audio-Language Models with LoRA for Precise Temporal Localization of Prolonged Exposure Therapy Elements

TL;DR

The paper tackles the bottleneck of PE therapy fidelity assessment by automatically localizing three key fidelity phases (P1, P2, P3) in session audio using a privacy-preserving, LoRA-finetuned Qwen2-Audio on 30-second windows. It introduces a soft-supervision labeling pipeline with LLM-generated timestamps and rater verification, and demonstrates that the approach achieves a mean absolute error of about 5.3 seconds on 308 sessions, competitive with human tolerance. The work analyzes the impact of window size and LoRA rank, revealing a context granularity trade-off and highlighting the practicality of automatic, fine-grained fidelity tracking for clinician training and quality assurance. It offers a scalable framework applicable to other protocol-driven conversational domains while preserving data privacy in clinical settings.

Abstract

Prolonged Exposure (PE) therapy is an effective treatment for post-traumatic stress disorder (PTSD), but evaluating therapist fidelity remains labor-intensive due to the need for manual review of session recordings. We present a method for the automatic temporal localization of key PE fidelity elements, identifying their start and stop times, directly from session audio and transcripts. Our approach fine-tunes a large pre-trained audio-language model, Qwen2-Audio, using Low-Rank Adaptation (LoRA) to process focused 30-second windows of audio-transcript input. Fidelity labels for three core protocol phases, therapist orientation (P1), imaginal exposure (P2), and post-imaginal processing (P3), are generated via LLM-based prompting and verified by trained raters. The model is trained to predict normalized boundary offsets using soft supervision guided by task-specific prompts. On a dataset of 308 real PE sessions, our best configuration (LoRA rank 8, 30s windows) achieves a mean absolute error (MAE) of 5.3s across tasks, within typical rater tolerance for timestamp review, enabling practical fidelity QC. We further analyze the effects of window size and LoRA rank, highlighting the importance of context granularity and model adaptation. This work introduces a privacy-preserving, scalable framework for fidelity tracking in PE therapy, with potential to support clinician training, supervision, and quality assurance.

Paper Structure

This paper contains 15 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Automated Evaluation of Psychotherapy
  4. Temporal Event Localization in Audio and Speech
  5. Large Pre-trained Audio-Language Models
  6. Multimodal Prompting & Few-Shot Learning
  7. Annotation and Supervision Strategies
  8. DATASET
  9. METHODOLOGY
  10. Problem Formulation
  11. Dataset and Soft-Supervision Labeling
  12. Model Architecture and Fine-tuning
  13. EXPERIMENTAL SETUP
  14. RESULTS AND DISCUSSION
  15. CONCLUSION

Figures (2)

  • Figure 1: Timeline of a 70-minute PE therapy session showing typical segments for three fidelity metrics: P1 (orientation to imaginal exposure), P2 (imaginal exposure), and P3 (processing). Durations vary across sessions; this structure represents a common clinical pattern. All durations are in minutes.
  • Figure 2: Overview of our fidelity-aligned modeling pipeline. A task-specific prompt guides QLoRA fine-tuning on Qwen2 Audio using 30-120s audio-transcript windows randomly sampled around annotated start/stop points. The model is trained to predict normalized temporal offsets using clinically validated or synthetic labels.