Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ISLE: An Intelligent Streaming Framework for High-Throughput AI Inference in Medical Imaging

Pranav Kulkarni, Sean Garin, Adway Kanhere, Eliot Siegel, Paul H. Yi, Vishwa S. Parekh

TL;DR

This work tackles bandwidth and compute bottlenecks in streaming medical images for AI inference by introducing ISLE, an intelligent streaming framework built on High-Throughput JPEG 2000 (HTJ2K) with progressive encoding. ISLE streams only the necessary sub-resolutions via a Progressive Encoder, a Stream Optimizer that selects the optimal decomposition, and a Progressive Decoder that reconstructs the chosen resolution, all without compromising AI performance. Across NIH, CheXpert, and MIMIC datasets (including DICOM), ISLE achieves data transmission reductions near 98%, decode-time reductions near 98%, and substantial throughput improvements, while maintaining AUROC comparable to full-resolution data. This approach enables scalable, cost-effective clinical AI inference at scale and could help democratize deployment across healthcare settings, with future work extending to 3D imaging and other AI tasks.

Abstract

As the adoption of Artificial Intelligence (AI) systems within the clinical environment grows, limitations in bandwidth and compute can create communication bottlenecks when streaming imaging data, leading to delays in patient care and increased cost. As such, healthcare providers and AI vendors will require greater computational infrastructure, therefore dramatically increasing costs. To that end, we developed ISLE, an intelligent streaming framework for high-throughput, compute- and bandwidth- optimized, and cost effective AI inference for clinical decision making at scale. In our experiments, ISLE on average reduced data transmission by 98.02% and decoding time by 98.09%, while increasing throughput by 2,730%. We show that ISLE results in faster turnaround times, and reduced overall cost of data, transmission, and compute, without negatively impacting clinical decision making using AI systems.

ISLE: An Intelligent Streaming Framework for High-Throughput AI Inference in Medical Imaging

TL;DR

This work tackles bandwidth and compute bottlenecks in streaming medical images for AI inference by introducing ISLE, an intelligent streaming framework built on High-Throughput JPEG 2000 (HTJ2K) with progressive encoding. ISLE streams only the necessary sub-resolutions via a Progressive Encoder, a Stream Optimizer that selects the optimal decomposition, and a Progressive Decoder that reconstructs the chosen resolution, all without compromising AI performance. Across NIH, CheXpert, and MIMIC datasets (including DICOM), ISLE achieves data transmission reductions near 98%, decode-time reductions near 98%, and substantial throughput improvements, while maintaining AUROC comparable to full-resolution data. This approach enables scalable, cost-effective clinical AI inference at scale and could help democratize deployment across healthcare settings, with future work extending to 3D imaging and other AI tasks.

Abstract

As the adoption of Artificial Intelligence (AI) systems within the clinical environment grows, limitations in bandwidth and compute can create communication bottlenecks when streaming imaging data, leading to delays in patient care and increased cost. As such, healthcare providers and AI vendors will require greater computational infrastructure, therefore dramatically increasing costs. To that end, we developed ISLE, an intelligent streaming framework for high-throughput, compute- and bandwidth- optimized, and cost effective AI inference for clinical decision making at scale. In our experiments, ISLE on average reduced data transmission by 98.02% and decoding time by 98.09%, while increasing throughput by 2,730%. We show that ISLE results in faster turnaround times, and reduced overall cost of data, transmission, and compute, without negatively impacting clinical decision making using AI systems.
Paper Structure (16 sections, 1 equation, 3 figures, 3 tables)

This paper contains 16 sections, 1 equation, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. High-Throughput JPEG 2000 (HTJ2K)
  4. Intelligent Streaming
  5. Progressive Encoder
  6. Stream Optimizer
  7. Progressive Decoder
  8. Datasets
  9. NIH Chest X-Ray 14
  10. CheXpert
  11. MIMIC-CXR-JPG
  12. Experiments
  13. Results
  14. Discussion
  15. Compliance with ethical standards
  16. ...and 1 more sections

Figures (3)

  • Figure 1: An overview of ISLE, our proposed intelligent streaming framework for high-throughput AI inference for clinical decision making at scale.
  • Figure 2: (a) Illustration of the progressive encoding of a chest x-ray and subsequent progressive decoding at different decompositions (i.e., sub-resolutions) by selecting different subsets of the image byte stream. (b) Suppose the same chest x-ray is encoded using JPEG (sequential) and HTJ2K (progressive) encoders, followed by decoding of partial byte streams from both formats. (a) The JPEG image results in an incomplete image whereas HTJ2K image is visible, albeit lower in resolution.
  • Figure 3: Mean AUROC scores on the original dataset and across each HTJ2K decomposition, including ISLE's optimal decomposition (hatched) on the (a) held-out internal NIH test set, (b) external CheXpert dataset, and (c) external MIMIC dataset. Comparison between the original dataset and ISLE's optimal decomposition is annotated. For all other comparisons, only statistically significant differences are annotated. (ns: $p \geq 0.05$, *: $p < 0.05$, **: $p < 0.01$, ***: $p < 0.001$)