Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Using Neural Networks for Data Cleaning in Weather Datasets

Jack R. P. Hanslope, Laurence Aitchison

TL;DR

This work trained a neural network to map from the wind field to the storm location, and found that this neural network trained only on the often noisy labels from IBTrACS had a denoising effect, and performed better than the IBTrACS labels themselves, as measured by human preferences.

Abstract

In climate science, we often want to compare across different datasets. Difficulties can arise in doing this due to inevitable mismatches that arise between observational and reanalysis data, or even between different reanalyses. This misalignment can raise problems for any work that seeks to make inferences about one dataset from another. We considered tropical cyclone location as an example task with one dataset providing atmospheric conditions (ERA5) and another providing storm tracks (IBTrACS). We found that while the examples often aligned well, there were a considerable proportion (around 25%) which were not well aligned. We trained a neural network to map from the wind field to the storm location; in this setting misalignment in the datasets appears as "label noise" (i.e. the labelled storm location does not correspond to the underlying wind field). We found that this neural network trained only on the often noisy labels from IBTrACS had a denoising effect, and performed better than the IBTrACS labels themselves, as measured by human preferences. Remarkably, this even held true for training points, on which we might have expected the network to overfit to the IBTrACS predictions.

Using Neural Networks for Data Cleaning in Weather Datasets

TL;DR

This work trained a neural network to map from the wind field to the storm location, and found that this neural network trained only on the often noisy labels from IBTrACS had a denoising effect, and performed better than the IBTrACS labels themselves, as measured by human preferences.

Abstract

In climate science, we often want to compare across different datasets. Difficulties can arise in doing this due to inevitable mismatches that arise between observational and reanalysis data, or even between different reanalyses. This misalignment can raise problems for any work that seeks to make inferences about one dataset from another. We considered tropical cyclone location as an example task with one dataset providing atmospheric conditions (ERA5) and another providing storm tracks (IBTrACS). We found that while the examples often aligned well, there were a considerable proportion (around 25%) which were not well aligned. We trained a neural network to map from the wind field to the storm location; in this setting misalignment in the datasets appears as "label noise" (i.e. the labelled storm location does not correspond to the underlying wind field). We found that this neural network trained only on the often noisy labels from IBTrACS had a denoising effect, and performed better than the IBTrACS labels themselves, as measured by human preferences. Remarkably, this even held true for training points, on which we might have expected the network to overfit to the IBTrACS predictions.
Paper Structure (13 sections, 2 figures, 1 table)

This paper contains 13 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Data Description
  5. ERA5
  6. IBTrACS
  7. Neural Network Setup
  8. Neural Network Architecture
  9. Training Setup
  10. Results
  11. Qualitative Results
  12. Quantitative Results
  13. Conclusion

Figures (2)

  • Figure 1: Wind fields for 4 hand chosen storms. IBTrACS location of storm centre marked by orange cross and probability predicted by the U-Net shown by the colourmap. Top left 1980-10-19 03:00; top right 1986-11-11 09:00; bottom left 1987-06-02 21:00; bottom right 1996-11-28 18:00.
  • Figure 2: Example of a wind field shown to the author for comparison.