RSF-Conv: Rotation-and-Scale Equivariant Fourier Parameterized Convolution for Retinal Vessel Segmentation

TL;DR

A rotation-and-scale equivariant Fourier parameterized convolution (RSF-Conv) specifically for retinal vessel segmentation is proposed and equivariant Fourier parameterized convolution (RSF-Conv) is provided to provide the corresponding equivariance analysis.

Abstract

Retinal vessel segmentation is of great clinical significance for the diagnosis of many eye-related diseases, but it is still a formidable challenge due to the intricate vascular morphology. With the skillful characterization of the translation symmetry existing in retinal vessels, convolutional neural networks (CNNs) have achieved great success in retinal vessel segmentation. However, the rotation-and-scale symmetry, as a more widespread image prior in retinal vessels, fails to be characterized by CNNs. Therefore, we propose a rotation-and-scale equivariant Fourier parameterized convolution (RSF-Conv) specifically for retinal vessel segmentation, and provide the corresponding equivariance analysis. As a general module, RSF-Conv can be integrated into existing networks in a plug-and-play manner while significantly reducing the number of parameters. For instance, we replace the traditional convolution filters in U-Net and Iter-Net with RSF-Convs, and faithfully conduct comprehensive experiments. RSF-Conv+U-Net and RSF-Conv+Iter-Net not only have slight advantages under in-domain evaluation, but more importantly, outperform all comparison methods by a significant margin under out-of-domain evaluation. It indicates the remarkable generalization of RSF-Conv, which holds greater practical clinical significance for the prevalent cross-device and cross-hospital challenges in clinical practice. To comprehensively demonstrate the effectiveness of RSF-Conv, we also apply RSF-Conv+U-Net and RSF-Conv+Iter-Net to retinal artery/vein classification and achieve promising performance as well, indicating its clinical application potential.

Figures (7)

• Figure 1: Illustration of the symmetries existing in retinal vessels and the performance of the rotation-and-scale equivariance. (a) A typical retinal image. (b) The corresponding retinal vessel image. (c) Local vascular patterns with the translation symmetry. (d) Local vascular patterns with the rotation symmetry. (e) Local vascular patterns with the scale symmetry. (f) Local vascular patterns with the rotation-and-scale symmetry. (g) Outputs of the randomly initialized traditional convolution filters. (h) Outputs of the randomly initialized RSF-Convs (ours).
• Figure 2: Illustration of the generating filters $\Psi$ at different orientations and scales. (a) The fixed basis functions. (b) The shared coefficient parameters. (c) Latent functions $\psi(x)$ at different orientations and scales. (d) Generated filters at different orientations and scales.
• Figure 3: Illustration of RSF-Conv, compared with the traditional convolution. (a)(b) The initial and intermediate equivariant convolution filters of RSF-Conv, with three input and nine output (i.e., 3 in/9 out) channels and 9 in/9 out channels, respectively. The kernels highlighted with the same colored box share the same weights. The discrete rotation group $R$ is {2$\pi$i/3, i=1, 2, 3}. The arrow orientations indicate the rotation angles of filters. The discrete scale group $S$ has three scale levels, which are indexed by the number of arrow shafts. The RSF-Conv filters with the same pattern are implemented by cyclically shifting along two dimensions, i.e., the rotation angles and the scale levels. (c)(d) The corresponding traditional convolution filters for comparison, which have no pre-designed structures.
• Figure 4: Some typical segmentation results under in-domain evaluation.
• Figure 5: Some typical segmentation results under out-of-domain evaluation.

Theorems & Definitions (4)

• Theorem
• Remark
• Theorem
• proof