Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HyDA: Hypernetworks for Test Time Domain Adaptation in Medical Imaging Analysis

Doron Serebro, Tammy Riklin-Raviv

TL;DR

Medical imaging models struggle with domain shift across acquisition protocols and devices, and traditional domain adaptation often needs target-domain data during training. HyDA addresses this by using a domain-conditioned hypernetwork that, via a domain encoder, generates external parameters for the primary network at test time, enabling interpolation to unseen domains. Across chest X-ray pathology classification and MRI brain age prediction, HyDA outperforms baselines and TTDA methods, demonstrating robust generalization without extensive target-domain data. The approach unifies domain representation learning with dynamic parameter generation, offering a practical path to deploy adaptable medical imaging models in real-world clinical settings.

Abstract

Medical imaging datasets often vary due to differences in acquisition protocols, patient demographics, and imaging devices. These variations in data distribution, known as domain shift, present a significant challenge in adapting imaging analysis models for practical healthcare applications. Most current domain adaptation (DA) approaches aim either to align the distributions between the source and target domains or to learn an invariant feature space that generalizes well across all domains. However, both strategies require access to a sufficient number of examples, though not necessarily annotated, from the test domain during training. This limitation hinders the widespread deployment of models in clinical settings, where target domain data may only be accessible in real time. In this work, we introduce HyDA, a novel hypernetwork framework that leverages domain characteristics rather than suppressing them, enabling dynamic adaptation at inference time. Specifically, HyDA learns implicit domain representations and uses them to adjust model parameters on-the-fly, effectively interpolating to unseen domains. We validate HyDA on two clinically relevant applications - MRI brain age prediction and chest X-ray pathology classification - demonstrating its ability to generalize across tasks and modalities. Our code is available at TBD.

HyDA: Hypernetworks for Test Time Domain Adaptation in Medical Imaging Analysis

TL;DR

Medical imaging models struggle with domain shift across acquisition protocols and devices, and traditional domain adaptation often needs target-domain data during training. HyDA addresses this by using a domain-conditioned hypernetwork that, via a domain encoder, generates external parameters for the primary network at test time, enabling interpolation to unseen domains. Across chest X-ray pathology classification and MRI brain age prediction, HyDA outperforms baselines and TTDA methods, demonstrating robust generalization without extensive target-domain data. The approach unifies domain representation learning with dynamic parameter generation, offering a practical path to deploy adaptable medical imaging models in real-world clinical settings.

Abstract

Medical imaging datasets often vary due to differences in acquisition protocols, patient demographics, and imaging devices. These variations in data distribution, known as domain shift, present a significant challenge in adapting imaging analysis models for practical healthcare applications. Most current domain adaptation (DA) approaches aim either to align the distributions between the source and target domains or to learn an invariant feature space that generalizes well across all domains. However, both strategies require access to a sufficient number of examples, though not necessarily annotated, from the test domain during training. This limitation hinders the widespread deployment of models in clinical settings, where target domain data may only be accessible in real time. In this work, we introduce HyDA, a novel hypernetwork framework that leverages domain characteristics rather than suppressing them, enabling dynamic adaptation at inference time. Specifically, HyDA learns implicit domain representations and uses them to adjust model parameters on-the-fly, effectively interpolating to unseen domains. We validate HyDA on two clinically relevant applications - MRI brain age prediction and chest X-ray pathology classification - demonstrating its ability to generalize across tasks and modalities. Our code is available at TBD.

Paper Structure

This paper contains 9 sections, 7 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Method
  4. Domain Conditioning Hypernetwork
  5. Loss Functions
  6. Experiments and Results
  7. Chest X-ray Pathology Classification
  8. Brain Age Prediction
  9. Conclusions

Figures (6)

  • Figure 1: The proposed HyDA framework (left) is composed of a hypernetwork $h$ (right), a primary network $\mathcal{P},$ and a domain classifier $\mathcal{D}.$ The hypernetwork generates weights and biases to the primary's head $\mathcal{P}_{head}$ based of the domain feature vector $X_D$ provided by the domain encoder $\mathcal{D}_{enc}$. Other weights in the system are internal and are learned via back-propagation using a regularized task dependent $\mathcal{L}_{RT}$, classification $\mathcal{L}_{CE}$ or/and multi-similarity $\mathcal{L}_{MSim}$ loss functions as illustrated by the dashed arrows.
  • Figure 2: All domains
  • Figure 3: w/o NIH
  • Figure 4: w/o CheXpert
  • Figure 5: w/o VinDr
  • ...and 1 more figures