DTC: A Deformable Transposed Convolution Module for Medical Image Segmentation

TL;DR

The paper tackles fixed-coordinate upsampling in medical image segmentation decoders and introduces Deformable Transposed Convolution (DTC), which learns dynamic sampling coordinates through a two-path upsampling design that combines convolution processing with a coordinate-generation branch using offsets and weights. The new coordinates are computed as $P_{n_{new}} = \lambda \times Offset \times Weight + P_{n_{Grid}}$ and sampled via $X' = grid\_sample(X, P_{n_{new}})$, with the final output being the sum of the conventional upsampling path and the deformable path. Across 2D (ISIC, BUSI) and 3D (BTCV) datasets and multiple architectures (UNet, SegMamba, SwinUNETR V2, nnUNet, nnMamba), DTC yields consistent Dice/NSD improvements and demonstrates compatibility with other upsampling methods, albeit with modest increases in parameters and computation. The results suggest DTC is a flexible, practical module that enhances decoder upsampling by adaptively focusing on relevant structures, potentially improving accuracy and robustness in clinical segmentation tasks.

Abstract

In medical image segmentation, particularly in UNet-like architectures, upsampling is primarily used to transform smaller feature maps into larger ones, enabling feature fusion between encoder and decoder features and supporting multi-scale prediction. Conventional upsampling methods, such as transposed convolution and linear interpolation, operate on fixed positions: transposed convolution applies kernel elements to predetermined pixel or voxel locations, while linear interpolation assigns values based on fixed coordinates in the original feature map. These fixed-position approaches may fail to capture structural information beyond predefined sampling positions and can lead to artifacts or loss of detail. Inspired by deformable convolutions, we propose a novel upsampling method, Deformable Transposed Convolution (DTC), which learns dynamic coordinates (i.e., sampling positions) to generate high-resolution feature maps for both 2D and 3D medical image segmentation tasks. Experiments on 3D (e.g., BTCV15) and 2D datasets (e.g., ISIC18, BUSI) demonstrate that DTC can be effectively integrated into existing medical image segmentation models, consistently improving the decoder's feature reconstruction and detail recovery capability.

Figures (9)

• Figure 1: Three common upsampling structures: (a) linear interpolation, (b) transposed convolution (also known as deconvolution), and (c) Dysample. Transposed convolution (b) allows channel adjustment by first zero-padding (indicated by white areas) and then convolving, while Dysample (c) maintains a consistent number of channels before and after upsampling. In Dysample, $g$ denotes the coordinate generation channel.
• Figure 2: Structure of Deformable Convolution (DC). $\alpha$ represents the number of information groups required by the convolution kernel to generate coordinates. K denotes the kernel size, B is the number of convolutions applied to the feature map, C is the number of feature channels, and G is the number of input groups used in multi-head attention.
• Figure 3: Structure of Deformable Transposed Convolution (DTC). The model consists of two components: (a) the upsampling method, typically implemented using linear interpolation or transposed convolution, and (b) the deformable transposed convolution component. In part (b), one path processes the feature information through convolution operations, while the other path generates offsets and weights via transposed convolution. These offsets and weight are used to compute new coordinates, $P_{n_{new}}$, which are combined with the convolution-generated features $\mathcal{X}$ through grid_sample to produce the new upsampled features, $\mathcal{X}'$. The outputs from both (a) and (b) are summed to form the final upsampling result. Here, W, H, N, and M represent the width, height, input channels, and output channels of the feature map, respectively. The upsampling scale is denoted by s, and $g$ represents the number of axes used for generating new coordinates through offsets and weights, with $g=2$ for 2D images and $g=3$ for 3D images. $P_{n}$ represents the original grid point coordinates, and $\lambda$ indicates the receptive field coefficient.
• Figure 4: Visualization comparing the effects of different upsampling methods on the same UNet model. The labels below each image indicate the model or the type of upsampling used, and the Dice score (%) for each segmentation is shown at the top-right corner.
• Figure 5: Example segmentation results on the ISIC (top row) and BUSI (bottom row) datasets. Each column corresponds to a different method, with white regions indicating the predicted segmentation masks. The numbers denote the Dice scores (%) for each prediction. Results also show the segmentation performance when DTC is introduced into different models.