A Study on Monthly Marine Heatwave Forecasts in New Zealand: An Investigation of Imbalanced Regression Loss Functions with Neural Network Models

TL;DR

This study reframes monthly marine heatwave forecasting as an imbalanced regression problem and evaluates a fully connected neural network across 12 New Zealand locations for lead times of 1–6 months. It systematically compares standard regression losses (MSE, MAE, Huber) with specialized losses (weighted MSE, focal-R, balanced MSE) and a novel scaling-weighted MSE designed to emphasize extreme events. Findings show that short-lead forecasts are more predictable, while specialized losses, particularly balanced MSE and scaling-weighted MSE, improve extreme-event predictions (MHWs and suspected MHWs) at many sites; long-lead forecasts remain challenging with occasional gains from certain losses. The work provides actionable guidance on loss-function selection for imbalanced regression in climate forecasts and introduces a loss tailored to balance normal versus extreme values, with implications for improved risk assessment of MHWs.

Abstract

Marine heatwaves (MHWs) are extreme ocean-temperature events with significant impacts on marine ecosystems and related industries. Accurate forecasts (one to six months ahead) of MHWs would aid in mitigating these impacts. However, forecasting MHWs presents a challenging imbalanced regression task due to the rarity of extreme temperature anomalies in comparison to more frequent moderate conditions. In this study, we examine monthly MHW forecasts for 12 locations around New Zealand. We use a fully-connected neural network and compare standard and specialized regression loss functions, including the mean squared error (MSE), the mean absolute error (MAE), the Huber, the weighted MSE, the focal-R, the balanced MSE, and a proposed scaling-weighted MSE. Results show that (i) short lead times (one month) are considerably more predictable than three- and six-month leads, (ii) models trained with the standard MSE or MAE losses excel at forecasting average conditions but struggle to capture extremes, and (iii) specialized loss functions such as the balanced MSE and our scaling-weighted MSE substantially improve forecasting of MHW and suspected MHW events. These findings underscore the importance of tailored loss functions for imbalanced regression, particularly in forecasting rare but impactful events such as MHWs.

Figures (54)

• Figure 1: 20 selected locations of interest for MHW forecasts in New Zealand. The SSTA time series of the locations in blue are both predictands and predictors. The SSTA time series of the locations in red are predictors only.
• Figure 2: SSTA observation and prediction scatter plots of a model that shows perfect underfitting (PU) on the training and test data respectively, which indicates no ability to predict.
• Figure 3: 12 selected locations with the corresponding results of monthly SSTA and MHW forecasts. The acronym "MHW" is "MHW" and "SMHW" is "suspected MHW", applied to the other figures in this study. The numbers following each type of forecasts are the numbers of lead months. If at least one of the model outperformed the persistence or fixed-lag model at one location in terms of the MSE for SSTAs, the CSI for MHWs, and the CSI 80 for suspected MHWs with the PUR not greater than 20%, the corresponding type of forecasts and the lead month are added to the location.
• Figure 4: 12 selected locations with the corresponding ranks of the loss functions based on the Friedman test with regards to monthly SSTA and MHW forecasts. Only the top three ranked predictors for one lead month SSTA, MHW, and SMHW at each location are selected. "&" denotes the same rank.
• Figure 5: Histograms of the SSTA time series of the six selected NZ locations: Bay of Plenty, Bank Peninsula, Chatham Island, Cape Reinga, Cook Strait, and Fiordland.