Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

StethoLM: Audio Language Model for Cardiopulmonary Analysis Across Clinical Tasks

Yishan Wang, Tsai-Ning Wang, Mathias Funk, Aaqib Saeed

TL;DR

StethoLM is presented, the first audio-language model specialized for cardiopulmonary auscultation, capable of performing instruction-driven clinical tasks across the full spectrum of auscultation analysis, and establishes a foundation for instruction-following AI systems in clinical auscultation.

Abstract

Listening to heart and lung sounds - auscultation - is one of the first and most fundamental steps in a clinical examination. Despite being fast and non-invasive, it demands years of experience to interpret subtle audio cues. Recent deep learning methods have made progress in automating cardiopulmonary sound analysis, yet most are restricted to simple classification and offer little clinical interpretability or decision support. We present StethoLM, the first audio-language model specialized for cardiopulmonary auscultation, capable of performing instruction-driven clinical tasks across the full spectrum of auscultation analysis. StethoLM integrates audio encoding with a medical language model backbone and is trained on StethoBench, a comprehensive benchmark comprising 77,027 instruction-response pairs synthesized from 16,125 labeled cardiopulmonary recordings spanning seven clinical task categories: binary classification, detection, reporting, reasoning, differential diagnosis, comparison, and location-based analysis. Through multi-stage training that combines supervised fine-tuning and direct preference optimization, StethoLM achieves substantial gains in performance and robustness on out-of-distribution data. Our work establishes a foundation for instruction-following AI systems in clinical auscultation.

StethoLM: Audio Language Model for Cardiopulmonary Analysis Across Clinical Tasks

TL;DR

StethoLM is presented, the first audio-language model specialized for cardiopulmonary auscultation, capable of performing instruction-driven clinical tasks across the full spectrum of auscultation analysis, and establishes a foundation for instruction-following AI systems in clinical auscultation.

Abstract

Listening to heart and lung sounds - auscultation - is one of the first and most fundamental steps in a clinical examination. Despite being fast and non-invasive, it demands years of experience to interpret subtle audio cues. Recent deep learning methods have made progress in automating cardiopulmonary sound analysis, yet most are restricted to simple classification and offer little clinical interpretability or decision support. We present StethoLM, the first audio-language model specialized for cardiopulmonary auscultation, capable of performing instruction-driven clinical tasks across the full spectrum of auscultation analysis. StethoLM integrates audio encoding with a medical language model backbone and is trained on StethoBench, a comprehensive benchmark comprising 77,027 instruction-response pairs synthesized from 16,125 labeled cardiopulmonary recordings spanning seven clinical task categories: binary classification, detection, reporting, reasoning, differential diagnosis, comparison, and location-based analysis. Through multi-stage training that combines supervised fine-tuning and direct preference optimization, StethoLM achieves substantial gains in performance and robustness on out-of-distribution data. Our work establishes a foundation for instruction-following AI systems in clinical auscultation.
Paper Structure (78 sections, 7 equations, 6 figures, 10 tables)

This paper contains 78 sections, 7 equations, 6 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Machine Learning for Cardiopulmonary Auscultation
  4. General-Purpose Audio-Language Models
  5. Multimodal Large Language Models for Healthcare
  6. Clinical Tasks in Medical AI: From Specialized to Generalist Systems
  7. Problem Formulation
  8. StethoBench
  9. Data Sources.
  10. In-Domain Datasets:
  11. Out-of-Domain Datasets:
  12. Out-of-Distribution Definition and Selection.
  13. Instruction-Response Pair Generation
  14. Single-Audio Analysis.
  15. Pairwise Comparison.
  16. ...and 63 more sections

Figures (6)

  • Figure 1: Overview of StethoLM and StethoBench. A. Automated benchmark creation pipeline, where off-the-shelf LLMs generate 77,027 task–response pairs from 16,125 cardiopulmonary recordings and associated annotations. B. Distribution of audio type and the examples of disease that StethoLM covers. C. StethoLM architecture integrating audio–text alignment, supervised fine-tuning for diverse clinical tasks.
  • Figure 2: Diverse clinical tasks supported by StethoLM. Instructions (left) represent realistic clinical queries, while responses (right) provide task-appropriate outputs ranging from binary decisions to complex diagnostic reasoning.
  • Figure 3: Zero-shot classification comparison with classification models. Accuracy across four binary tasks comparing zero-shot inference (StethoLM via report generation + text similarity) against supervised linear probing (AudioMAE, CLAP, Opera-CE). Tasks include COVID-19 detection from exhalation/cough recordings and COPD screening from lung sounds.
  • Figure 4: Attention patterns revealing audio grounding during response generation. Heatmap visualizes how generated response tokens (vertical axis) attend to key tokens (horizontal axis). Black solid lines mark the audio-instruction boundary; red dashed lines mark where response generation begins.
  • Figure 5: Qualitative analysis of StethoLM's clinical reasoning capabilities. Representative outputs demonstrate the model's ability to generate clinically coherent responses across diverse auscultation tasks.
  • ...and 1 more figures