Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Signals Generate Clinical Notes in the Wild

Jathurshan Pradeepkumar, Zheng Chen, Jimeng Sun

TL;DR

This work tackles the labor-intensive task of producing clinical EEG reports from long-duration recordings by introducing CELM, the first EEG-to-language foundation model. CELM integrates a pretrained EEG encoder with a large language model through novel epoch-level tokenization, sequence-aware alignment, and prompt fusion to achieve multi-scale, end-to-end report generation. On a large-scale Harvard EEG benchmark, CELM substantially outperforms strong unimodal baselines, delivering $70 ext{–}95 ext{ extpercent}$ relative improvements with patient history and $0.43$–$0.52$ in zero-context generation scores, while also highlighting the key challenges in representing long sequences and rare clinical events. The work provides a scalable benchmark pipeline and emphasizes the potential for automated, clinically grounded EEG-to-language systems, though it notes limitations in evaluation metrics, memory constraints, and the need for careful validation before real-world deployment.

Abstract

Generating clinical reports that summarize abnormal patterns, diagnostic findings, and clinical interpretations from long-term EEG recordings remains labor-intensive. We curate a large-scale clinical EEG dataset with $9{,}922$ reports paired with approximately $11{,}000$ hours of EEG recordings from $9{,}048$ patients. We therefore develop CELM, the first clinical EEG-to-Language foundation model capable of summarizing long-duration, variable-length EEG recordings and performing end-to-end clinical report generation at multiple scales, including recording description, background activity, epileptiform abnormalities, events/seizures, and impressions. Experimental results show that, with patient history supervision, our method achieves $70\%$--$95\%$ average relative improvements in standard generation metrics (e.g., ROUGE-1 and METEOR) from $0.2$--$0.3$ to $0.4$--$0.6$. In the zero-shot setting without patient history, CELM attains generation scores in the range of $0.43$--$0.52$, compared to baselines of $0.17$--$0.26$. CELM integrates pretrained EEG foundation models with language models to enable scalable multimodal learning. We release our model and benchmark construction pipeline at [URL].

Neural Signals Generate Clinical Notes in the Wild

TL;DR

This work tackles the labor-intensive task of producing clinical EEG reports from long-duration recordings by introducing CELM, the first EEG-to-language foundation model. CELM integrates a pretrained EEG encoder with a large language model through novel epoch-level tokenization, sequence-aware alignment, and prompt fusion to achieve multi-scale, end-to-end report generation. On a large-scale Harvard EEG benchmark, CELM substantially outperforms strong unimodal baselines, delivering relative improvements with patient history and – in zero-context generation scores, while also highlighting the key challenges in representing long sequences and rare clinical events. The work provides a scalable benchmark pipeline and emphasizes the potential for automated, clinically grounded EEG-to-language systems, though it notes limitations in evaluation metrics, memory constraints, and the need for careful validation before real-world deployment.

Abstract

Generating clinical reports that summarize abnormal patterns, diagnostic findings, and clinical interpretations from long-term EEG recordings remains labor-intensive. We curate a large-scale clinical EEG dataset with reports paired with approximately hours of EEG recordings from patients. We therefore develop CELM, the first clinical EEG-to-Language foundation model capable of summarizing long-duration, variable-length EEG recordings and performing end-to-end clinical report generation at multiple scales, including recording description, background activity, epileptiform abnormalities, events/seizures, and impressions. Experimental results show that, with patient history supervision, our method achieves -- average relative improvements in standard generation metrics (e.g., ROUGE-1 and METEOR) from -- to --. In the zero-shot setting without patient history, CELM attains generation scores in the range of --, compared to baselines of --. CELM integrates pretrained EEG foundation models with language models to enable scalable multimodal learning. We release our model and benchmark construction pipeline at [URL].
Paper Structure (29 sections, 2 equations, 5 figures, 4 tables)

This paper contains 29 sections, 2 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. EEG-to-Language Modeling.
  4. EEG-Report Benchmark Construction
  5. Clinical Report Structuring
  6. EEG Preprocessing
  7. CELM: Clinical EEG Language Model
  8. Architecture and Forward Process
  9. Epoch-Aggregated Tokenization
  10. Sequence-Aware Alignment
  11. Sequence Context Alignment (SCA).
  12. Sequence Context Compression (SCC).
  13. Prompt Fusion and Generation
  14. Experiments and Results
  15. Datasets.
  16. ...and 14 more sections

Figures (5)

  • Figure 1: Overview of our framework. (a) EEG–Report benchmark construction pipeline, including clinical report structuring (Section \ref{['subsection:3.1']}), matching reports to EEG sessions, and examples of standardized report sections. (b) The proposed Clinical EEG Language Model (CELM) comprises (b.1) Epoch-Aggregated Tokenization, (b.2) Sequence-Aware Alignment, and (b.3) Prompt Fusion and Generation.
  • Figure 2: Section-wise comparison of report generation performance between CELM and the best-performing baselines from different LLM families.
  • Figure 3: (a) Report generation performance of different projector variants. (b) Training dynamics of each variant, including training loss, validation loss, and perplexity curves.
  • Figure 4: Examples of generated reports on S0002. Comparisons between the unimodal baseline (text + EEG features), the linear-projector alignment variant, CELM, and the ground-truth reports. (a) EEG description/details; (b) Impression/interpretation.
  • Figure 5: Dataset statistics of the filtered and constructed EEG-Report Benchmark used in our study