Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploiting Intermediate Reconstructions in Optical Coherence Tomography for Test-Time Adaption of Medical Image Segmentation

Thomas Pinetz, Veit Hucke, Hrvoje Bogunovic

TL;DR

IRTTA is proposed to exploit intermediate representations of a frozen downstream network via a modulator network that conditions on the current reconstruction timescale by adapting the normalization-layer parameters of a frozen downstream network via a modulator network that conditions on the current reconstruction timescale.

Abstract

Primary health care frequently relies on low-cost imaging devices, which are commonly used for screening purposes. To ensure accurate diagnosis, these systems depend on advanced reconstruction algorithms designed to approximate the performance of high-quality counterparts. Such algorithms typically employ iterative reconstruction methods that incorporate domain-specific prior knowledge. However, downstream task performance is generally assessed using only the final reconstructed image, thereby disregarding the informative intermediate representations generated throughout the reconstruction process. In this work, we propose IRTTA to exploit these intermediate representations at test-time by adapting the normalization-layer parameters of a frozen downstream network via a modulator network that conditions on the current reconstruction timescale. The modulator network is learned during test-time using an averaged entropy loss across all individual timesteps. Variation among the timestep-wise segmentations additionally provides uncertainty estimates at no extra cost. This approach enhances segmentation performance and enables semantically meaningful uncertainty estimation, all without modifying either the reconstruction process or the downstream model.

Exploiting Intermediate Reconstructions in Optical Coherence Tomography for Test-Time Adaption of Medical Image Segmentation

TL;DR

IRTTA is proposed to exploit intermediate representations of a frozen downstream network via a modulator network that conditions on the current reconstruction timescale by adapting the normalization-layer parameters of a frozen downstream network via a modulator network that conditions on the current reconstruction timescale.

Abstract

Primary health care frequently relies on low-cost imaging devices, which are commonly used for screening purposes. To ensure accurate diagnosis, these systems depend on advanced reconstruction algorithms designed to approximate the performance of high-quality counterparts. Such algorithms typically employ iterative reconstruction methods that incorporate domain-specific prior knowledge. However, downstream task performance is generally assessed using only the final reconstructed image, thereby disregarding the informative intermediate representations generated throughout the reconstruction process. In this work, we propose IRTTA to exploit these intermediate representations at test-time by adapting the normalization-layer parameters of a frozen downstream network via a modulator network that conditions on the current reconstruction timescale. The modulator network is learned during test-time using an averaged entropy loss across all individual timesteps. Variation among the timestep-wise segmentations additionally provides uncertainty estimates at no extra cost. This approach enhances segmentation performance and enables semantically meaningful uncertainty estimation, all without modifying either the reconstruction process or the downstream model.
Paper Structure (15 sections, 4 equations, 3 figures, 7 tables)

This paper contains 15 sections, 4 equations, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Problem Formulation
  4. Iterative Reconstruction via Diffusion
  5. Test-Time Modulation Network
  6. Optimization and Uncertainty Estimation
  7. Results
  8. Experimental Setup
  9. Dataset and Preprocessing
  10. Implementation Details
  11. Baselines and Comparison Methods
  12. Quantitative Analysis
  13. Ablation Study
  14. Uncertainty visualization
  15. Conclusion

Figures (3)

  • Figure 1: A schematic overview of our proposed method IRTTA. The differences between the reconstructed $x_0,\ldots,x_{S-1}$. Initially, the weights and biases of the final layer in $g_\Psi$ is set to $0$, which is highlighted with bold arrows. Hence, the frozen backbone $f_\theta$ retains its performance at initialization and is adapted without labels.
  • Figure 2: Visual comparison of downstream results using different methods. The top and bottom two rows show examples from the Cirrus and Topcon datasets respectively.
  • Figure 3: Visual comparison of uncertainty estimation compared to the baseline, visualized by showing the entropy of the prediction. The uncertainty now shows semantically meaningful information instead of the boundary of the initial segmentation.