LubDubDecoder: Bringing Micro-Mechanical Cardiac Monitoring to Hearables

TL;DR

This paper introduces LubDubDecoder, a hearable-based system that converts the in-ear speaker into an acoustic sensor to reconstruct fine micro-mechanical cardiac signals (SCG and GCG) from ear-based lub-dub sounds. It achieves cross-device generalization via a zero-effort normalization and a lightweight calibration step, and demonstrates robust performance across remounting, music playback, and real-world conditions in a feasibility study with 25 participants. The core contributions include a cross-modal autoencoder for ear sounds to SCG/GCG reconstruction, an automated fiducial-point labeling method, motion artifact removal, and cross-user and cross-device generalization strategies, yielding high waveform correlation (0.88–0.95) and low timing error (median near 0–0.5% of the cardiac cycle). The work advances ubiquitous, unobtrusive cardiac monitoring by enabling micro-cardiac event timing from everyday hearables, which can support early screening and continuous health insights in real-world settings.

Abstract

We present LubDubDecoder, a system that enables fine-grained monitoring of micro-cardiac vibrations associated with the opening and closing of heart valves across a range of hearables. Our system transforms the built-in speaker, the only transducer common to all hearables, into an acoustic sensor that captures the coarse "lub-dub" heart sounds, leverages their shared temporal and spectral structure to reconstruct the subtle seismocardiography (SCG) and gyrocardiography (GCG) waveforms, and extract the timing of key micro-cardiac events. In an IRB-approved feasibility study with 25 users, our system achieves correlations of 0.88-0.95 compared to chest-mounted reference measurements in within-user and cross-user evaluations, and generalizes to unseen hearables using a zero-effort adaptation scheme with a correlation of 0.91. Our system is robust across remounting sessions and music playback.

Figures (21)

• Figure 1: Cardiac signal timing diagram. Each heartbeat begins with the electrical depolarization of the ventricles, measurable by the ECG. This is followed by the mitral valve closing which creates the "lub" (S1) sound of the heart, which generates an acoustic signal detectable at the ear. Later in the cardiac cycle, the aortic valve closes, producing the "dub" (S2) sound. The heart’s mechanical motion can be captured at using SCG and GCG.
• Figure 2: Challenge of conventional IMU-based micro-cardiac measurements. Differences in sensor placement lead to variations in waveforms, making comparisons across repeated measurements challenging. Precise and consistent placement is difficult to ensure when measurements are performed by lay users outside clinical settings. Each waveform corresponds to a cycle of 800 ms, and amplitudes are normalized to their own maximum.
• Figure 3: Dataset collection setup. (a) Ear-based cardiac sounds are measured using the microphone or speaker of a hearable; mechanical cardiac vibrations are measured at the left lower sternal border around the heart using a smartphone IMU. (b) Hearables used for data collection span a range of device types.
• Figure 4: Effect of device remounting on cardiac signals. Within a single session, cardiac signals show similar morphology across cycles ($n=60$ cycles). After remounting the hearable and smartphone, ear-based cardiac sounds and micro-mechanical signals maintain comparable waveform shapes, with a modest increase in variability across cycles. Colored opaque line is mean signal across cardiac cycles, all cardiac cycles are overlaid in translucent gray. Each waveform corresponds to a cycle of 800 ms, and amplitudes are normalized to their own maximum.
• Figure 5: Effect of individual physiology on cardiac signal variability. Waveform variability is presented for a random subset of $n=6$ subjects from our human subjects study showing differences in ear-based cardiac sounds, SCG, and GCG signals ($n=60$ cycles). Solid opaque lines represents the mean across all cardiac cycles within each subject, all cardiac cycles are overlaid in translucent color. Each waveform corresponds to a cycle of 800 ms, and amplitudes are normalized to their own maximum.