Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Concise Tiling Strategy for Preserving Spatial Context in Earth Observation Imagery

Ellianna Abrahams, Tasha Snow, Matthew R. Siegfried, Fernando Pérez

TL;DR

Flip-n-Slide outperforms the previous state-of-the-art augmentation routines for tiled data in all evaluation metrics and enhances the generalizability of the training set without misrepresenting the true data distribution.

Abstract

We propose a new tiling strategy, Flip-n-Slide, which has been developed for specific use with large Earth observation satellite images when the location of objects-of-interest (OoI) is unknown and spatial context can be necessary for class disambiguation. Flip-n-Slide is a concise and minimalistic approach that allows OoI to be represented at multiple tile positions and orientations. This strategy introduces multiple views of spatio-contextual information, without introducing redundancies into the training set. By maintaining distinct transformation permutations for each tile overlap, we enhance the generalizability of the training set without misrepresenting the true data distribution. Our experiments validate the effectiveness of Flip-n-Slide in the task of semantic segmentation, a necessary data product in geophysical studies. We find that Flip-n-Slide outperforms the previous state-of-the-art augmentation routines for tiled data in all evaluation metrics. For underrepresented classes, Flip-n-Slide increases precision by as much as 15.8%.

A Concise Tiling Strategy for Preserving Spatial Context in Earth Observation Imagery

TL;DR

Flip-n-Slide outperforms the previous state-of-the-art augmentation routines for tiled data in all evaluation metrics and enhances the generalizability of the training set without misrepresenting the true data distribution.

Abstract

We propose a new tiling strategy, Flip-n-Slide, which has been developed for specific use with large Earth observation satellite images when the location of objects-of-interest (OoI) is unknown and spatial context can be necessary for class disambiguation. Flip-n-Slide is a concise and minimalistic approach that allows OoI to be represented at multiple tile positions and orientations. This strategy introduces multiple views of spatio-contextual information, without introducing redundancies into the training set. By maintaining distinct transformation permutations for each tile overlap, we enhance the generalizability of the training set without misrepresenting the true data distribution. Our experiments validate the effectiveness of Flip-n-Slide in the task of semantic segmentation, a necessary data product in geophysical studies. We find that Flip-n-Slide outperforms the previous state-of-the-art augmentation routines for tiled data in all evaluation metrics. For underrepresented classes, Flip-n-Slide increases precision by as much as 15.8%.
Paper Structure (12 sections, 4 figures, 2 tables, 1 algorithm)

This paper contains 12 sections, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Tiling and Augmentation Method
  3. Mathematical Formalism for the Flip-n-Slide Algorithm
  4. Hyperparameter Selection for Tiling and Transformation
  5. Experimental Setup
  6. Experimental Results
  7. Performance Improvement for Underrepresented Classes
  8. Conclusions
  9. Appendix
  10. The Flip-n-Slide Algorithm
  11. Computational Cost Analysis
  12. Ablation Studies

Figures (4)

  • Figure 1: Flip-n-Slide's tile overlap strategy creates eight overlapping tiles for any image region more than a 75% tile threshold away from the overall image edge. Three tiling strategies, shown in false color to illustrate overlap, are visualized here. a) Tiles do not overlap. b) The conventional tile overlap strategy, shown at the recommended 50% overlap. c) Flip-n-Slide includes more tile overlaps, capturing more OoI tile position views for the training set.
  • Figure 2: In Earth observation, classes can be extremely imbalanced as is shown here for Ellesmere Island, Nunavut, CA. a) Labels from the Land Cover of Canada dataset latifovic_2020 overlain on the corresponding Landsat satellite imagery eros_2013. The legend shows the relative class distribution. Image tiles showing (b) an over-represented class (snow and ice) and (c) an under-represented class (lichen and moss) with the binary class masks from the ground truth data set, segmentation after training using the Flip-n-Slide tiling algorithm, and segmentation after training using the conventional tiling algorithm (50% overlap). Although both algorithms perform well for the over-represented class case, Flip-n-Slide is more precise, by up to 15.8%, than the conventional strategy (50% tile overlap).
  • Figure 3: Ablation studies on varying tile size confirm that our strategy leads to expected model behavior. Here we show mAP for each study, averaged over three test runs with the standard deviation shown in grey. Flip-n-Slide outperforms the conventional 50% overlap strategy even at smaller input tile sizes.
  • Figure 4: To minimize redundancy in the Flip-n-Slide strategy, each overlapping tile is uniquely permuted with a distinct, physically-realistic transformation, as shown here. Previous strategies have not employed overlap-specific transformations; any augmentations have been applied across all tiles or at random. Tiles are shown in false color to illustrate overlapping areas. Transparency indicates areas that do not overlap with the blue tile shown here. They overlap with neighboring blue tiles instead.