Compact Multimodal Language Models as Robust OCR Alternatives for Noisy Textual Clinical Reports

TL;DR

The paper tackles the problem of digitizing noisy, smartphone-captured clinical documents by evaluating compact multimodal language models (MLLMs) as privacy-preserving OCR alternatives. It benchmarks eight systems across four families on a private corpus of 340 obstetric ultrasound reports from India, using CER, WER, and a numeric accuracy metric $N_{acc}$ to assess both general transcription and numeric fidelity. Results show that compact MLLMs outperform classical OCR and neural pipelines in accuracy and preserve numerical content with $N_{acc} > 0.92$, albeit with higher hardware requirements and latency. The study provides practical guidance for on-premises healthcare digitization, highlights the robustness of MLLMs to common image degradations, and identifies limitations and future directions for structured extraction and uncertainty-aware workflows.

Abstract

Digitization of medical records often relies on smartphone photographs of printed reports, producing images degraded by blur, shadows, and other noise. Conventional OCR systems, optimized for clean scans, perform poorly under such real-world conditions. This study evaluates compact multimodal language models as privacy-preserving alternatives for transcribing noisy clinical documents. Using obstetric ultrasound reports written in regionally inflected medical English common to Indian healthcare settings, we compare eight systems in terms of transcription accuracy, noise sensitivity, numeric accuracy, and computational efficiency. Compact multimodal models consistently outperform both classical and neural OCR pipelines. Despite higher computational costs, their robustness and linguistic adaptability position them as viable candidates for on-premises healthcare digitization.

Figures (9)

• Figure 1: Critical-difference diagram of mean ranks computed from per-document CER and WER values. Lower ranks indicate better performance; groups connected by a bar do not differ significantly at $\alpha{=}0.05$.
• Figure 2: Per-model correlations between OCR character error rate and noise indicators after Benjamini-Hochberg correction for multiple comparisons. Rows correspond to OCR models and columns to noise metrics. Each cell reports Spearman’s $\rho$ with the corresponding raw $p$-value and FDR-adjusted $q$-value; asterisks mark correlations significant at $q \le 0.05$. Warmer colors indicate stronger positive associations, while cooler colors denote negative correlations.
• Figure 3: Correlations between no-reference image quality assessment (NR-IQA) metrics and manually annotated noise indicators. Rows correspond to NR-IQA metrics and columns to noise dimensions.
• Figure 4: Representative document fragments showing typical noise factors observed in the dataset. These artifacts arise from handheld capture conditions.
• Figure 5: Per-model correlations between OCR word error rate and noise indicators after Benjamini-Hochberg correction for multiple comparisons. Rows correspond to OCR models and columns to noise metrics. Each cell reports Spearman’s $\rho$ with the corresponding raw $p$-value and FDR-adjusted $q$-value; asterisks mark correlations significant at $q \le 0.05$. Warmer colors indicate stronger positive associations, while cooler colors denote negative correlations.