Estimating Physical Information Consistency of Channel Data Augmentation for Remote Sensing Images

TL;DR

This work addresses whether channel-based data augmentations preserve the physical information in remote sensing imagery. It introduces a deviation-based metric that compares augmented and unaugmented pixel signatures within time-series, producing $S_{ ext{noaug}}$ and $S_{ ext{aug}}$ to gauge physical consistency. Experiments on BigEarthNet-S2 show that some augmentations (e.g., contrast, Gaussian blur, Gaussian noise, posterize, sharpness, solarize) largely preserve physical information, while grayscale and strong brightness changes can violate it and do not improve baseline performance. The findings offer a practical framework to select augmentation strategies that balance realism and generalization, with implications for SSL and supervised RS tasks and future work extending to other modalities.

Abstract

The application of data augmentation for deep learning (DL) methods plays an important role in achieving state-of-the-art results in supervised, semi-supervised, and self-supervised image classification. In particular, channel transformations (e.g., solarize, grayscale, brightness adjustments) are integrated into data augmentation pipelines for remote sensing (RS) image classification tasks. However, contradicting beliefs exist about their proper applications to RS images. A common point of critique is that the application of channel augmentation techniques may lead to physically inconsistent spectral data (i.e., pixel signatures). To shed light on the open debate, we propose an approach to estimate whether a channel augmentation technique affects the physical information of RS images. To this end, the proposed approach estimates a score that measures the alignment of a pixel signature within a time series that can be naturally subject to deviations caused by factors such as acquisition conditions or phenological states of vegetation. We compare the scores associated with original and augmented pixel signatures to evaluate the physical consistency. Experimental results on a multi-label image classification task show that channel augmentations yielding a score that exceeds the expected deviation of original pixel signatures can not improve the performance of a baseline model trained without augmentation.

Figures (2)

• Figure 1: Qualitative overview for calculating $S_{\text{aug}}$ (or $S_{\text{noaug}}$). Step 1: Construct a set $D$ of $N$ time series $\mathbf{t}_i$. In the case of optical data, filter out all cloudy images. Step 2: Per time series $\mathbf{t}_i$ select a homogeneous $k \times k$ sub-area that is not impacted by any land use land cover changes over time to generate mask $\mathbf{b}_i$. Step 3: For each image $\mathbf{t}_{i,\tau}$ in $\mathbf{t}_i$ compute the respective pixel signature by averaging over the selected sub-area per band. Then, for each signature (red) from an augmented image for $S_{\text{aug}}$ (or unaugmented image for $S_{\text{noaug}}$), compute the distance to the closest unaugmented signature (green) within the same time series.
• Figure 2: Analysis of the degree of physical consistency of channel augmentation techniques. The score $S_{\text{noaug}}$ is plotted as the dashed gray line with its standard deviation $\sigma$ depicted in light gray. The solid line represents the score for $S_{\text{aug}}$. If the training performance of applying the respective augmentation technique with the corresponding maximum magnitude during training ($\text{mAP}_{\text{aug}}$) exceeds the training performance without applying any augmentation techniques ($\text{mAP}_{\text{noaug}}$), the solid line is depicted in blue. If the opposite holds true, the solid line is depicted in red. The considered augmentation techniques are: (a) Brightness; (b) Contrast; (c) Gaussian Blur; (d) Gaussian Noise; (e) Grayscale; (f) Posterize; (g) Sharpness; and (h) Solarize.