Encoder-Free ECG-Language Models

TL;DR

The paper tackles the challenge of scalable ECG interpretation by extending encoder-free design principles from vision-language to ECG-language models. It introduces ELF, which maps flattened ECG signals into the LLM embedding space via a single trainable projection, eliminating the need for a pretrained ECG encoder. Across five ECG datasets and multiple backbones, ELF achieves competitive performance with encoder-based ELMs, with ablations suggesting minimal gains from additional inductive biases. Perturbation analyses reveal that language priors often drive performance, highlighting limitations in current ECG benchmarks and the need for more demanding evaluation settings. The study underscores a practical simplification in ECG-language modeling and advocates for improved datasets to realize true ECG understanding in ELMs.

Abstract

ECG-Language Models (ELMs) extend recent progress in Multimodal Large Language Models (MLLMs) to automated ECG interpretation. However, most ELMs follow Vision-Language Model (VLM) designs and depend on pretrained ECG encoders, adding architectural and training complexity. Inspired by encoder-free VLMs, we introduce ELF, an encoder-free ELM that replaces the ECG encoder with a single projection layer trained jointly with the LLM. Across five datasets, ELF matches or exceeds state-of-the-art ELMs that use far more complex encoders and training pipelines. We also test whether adding architectural biases to ELF improves performance and find that the single linear projection remains competitive. Finally, we show that ELF, and potentially other ELMs, often rely more on benchmark artifacts and language priors than ECG-derived information, highlighting limitations in current evaluation practices and ELM design. All data and code is available at https://github.com/willxxy/ECG-Bench.

Figures (4)

• Figure 1: The Architecture of ELF. Given an ECG signal $X \in \mathbb{R}^{C \times L}$, where $C$ is the number of leads and $L$ the signal length, we flatten it to $x \in \mathbb{R}^{CL}$ and project it into a $d$-dimensional latent vector $z = Wx + b$, with $W \in \mathbb{R}^{d \times (CL)}$ and $b \in \mathbb{R}^{d}$. The projected ECG embedding is inserted into the textual query embeddings $E \in \mathbb{R}^{T \times d}$ and processed autoregressively by an LLM to generate outputs $y_{1:T}$.
• Figure 2: Averaged accuracy across three random seeds for each ELM on three datasets. All ELMs use the Llama-3.2-1B-Instruct backbone; best results are highlighted in yellow.
• Figure 3: We compare ELF's performance against strong ELMs with dedicated encoders when evaluating across increasing steps trained. All results are averaged across three random seeds and trained/evaluated on the PTB-XL ECG-QA dataset.
• Figure 4: We train and inference ELF under three training and inference conditions: 1) ECG Signal, 2) Zeros Tensor, and 3) Only Text. All reported results are averaged across three random seeds on the PTB-XL ECG-QA dataset.