Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-scale gridded Gabor attention for cirrus segmentation

Felix Richards, Adeline Paiement, Xianghua Xie, Elisabeth Sola, Pierre-Alain Duc

TL;DR

The paper tackles the problem of segmenting large-scale atmospheric contaminants in astronomical imagery, where global context and texture orientation are both crucial. It introduces a gridded multi-scale attention framework augmented by a Gabor orientation module (tri-attention) and a consensus-based Super-Majority Loss to handle uncertain labels, integrated into a final segmentation model. Empirical results on MATLAS cirrus data and synthetic images show significant performance gains over baselines, with state-of-the-art results on SWIMSEG cloud segmentation; the approach is computationally efficient due to tile-based processing and scale-sharing, enabling near one-pass whole-image segmentation. The work offers practical impact for processing imaging data from astronomical instruments and lays groundwork for future deep generative contaminant removal methods.

Abstract

In this paper, we address the challenge of segmenting global contaminants in large images. The precise delineation of such structures requires ample global context alongside understanding of textural patterns. CNNs specialise in the latter, though their ability to generate global features is limited. Attention measures long range dependencies in images, capturing global context, though at a large computational cost. We propose a gridded attention mechanism to address this limitation, greatly increasing efficiency by processing multi-scale features into smaller tiles. We also enhance the attention mechanism for increased sensitivity to texture orientation, by measuring correlations across features dependent on different orientations, in addition to channel and positional attention. We present results on a new dataset of astronomical images, where the task is segmenting large contaminating dust clouds.

Multi-scale gridded Gabor attention for cirrus segmentation

TL;DR

The paper tackles the problem of segmenting large-scale atmospheric contaminants in astronomical imagery, where global context and texture orientation are both crucial. It introduces a gridded multi-scale attention framework augmented by a Gabor orientation module (tri-attention) and a consensus-based Super-Majority Loss to handle uncertain labels, integrated into a final segmentation model. Empirical results on MATLAS cirrus data and synthetic images show significant performance gains over baselines, with state-of-the-art results on SWIMSEG cloud segmentation; the approach is computationally efficient due to tile-based processing and scale-sharing, enabling near one-pass whole-image segmentation. The work offers practical impact for processing imaging data from astronomical instruments and lays groundwork for future deep generative contaminant removal methods.

Abstract

In this paper, we address the challenge of segmenting global contaminants in large images. The precise delineation of such structures requires ample global context alongside understanding of textural patterns. CNNs specialise in the latter, though their ability to generate global features is limited. Attention measures long range dependencies in images, capturing global context, though at a large computational cost. We propose a gridded attention mechanism to address this limitation, greatly increasing efficiency by processing multi-scale features into smaller tiles. We also enhance the attention mechanism for increased sensitivity to texture orientation, by measuring correlations across features dependent on different orientations, in addition to channel and positional attention. We present results on a new dataset of astronomical images, where the task is segmenting large contaminating dust clouds.
Paper Structure (13 sections, 1 equation, 3 figures, 3 tables)

This paper contains 13 sections, 1 equation, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Background on Multi-scale Attention
  4. Multi-scale Gridded Attention
  5. Gabor Attention
  6. Super-Majority Loss function
  7. Final Segmentation Model
  8. Results
  9. Data
  10. Comparative Analyses on Proposed Techniques
  11. Proposed Cirrus Segmentation
  12. Cloud segmentation in natural images
  13. Conclusion

Figures (3)

  • Figure 1: Cirrus dust of various strengths (top), with uncertain annotations (middle) and predictions (bottom).
  • Figure 2: Proposed MS gridded attention.
  • Figure 3: Proposed Gabor attention operator. $G$ is number of modulating Gabor filters, $N$ is the product of other axes. $\bigotimes$ denotes matrix multiplication, and $\bigoplus$ element-wise addition.