Estimating the distance at which narwhal $(\textit{Monodon monoceros})$ respond to disturbance: a penalized threshold hidden Markov model

TL;DR

We address estimating disturbance thresholds in narwhal responses to vessel disturbance using telemetry data. The authors introduce a lasso-penalized threshold hidden Markov model (THMM) that employs a smooth logistic approximation to a threshold and a qREML-based procedure to select the penalty strength, enabling simultaneous threshold estimation and assessment of meaningful disturbance, with the threshold on the standardized covariate given by $1/\hat{\beta}_0$. Simulations show controlled false positives and accurate threshold recovery, and application to narwhal data reveals responses up to about $4$ km from vessels, with land between vessel and whale attenuating exposure. The framework is broadly applicable to other species and stimuli, offering a scalable tool to inform disturbance-mitigation policies.

Abstract

Understanding behavioural responses to disturbances is vital for wildlife conservation. For example, in the Arctic, the decrease in sea ice has opened new shipping routes, increasing the need for impact assessments that quantify the distance at which marine mammals react to vessel presence. This information can then guide targeted mitigation policies, such as vessel slow-down regulations and delineation of avoidance areas. Using telemetry data to determine distances linked to deviations from normal behaviour requires advanced statistical models, such as threshold hidden Markov models (THMMs). While these are powerful tools, they do not assess whether the estimated threshold reflects a meaningful behavioural shift. We introduce a lasso-penalized THMM that builds on computationally efficient methods to impose penalties on HMMs and present a new, efficient penalized quasi-restricted maximum-likelihood estimator. Our framework is capable of estimating thresholds and assessing whether the disturbance effects are meaningful. With simulations, we demonstrate that our lasso method effectively shrinks spurious threshold effects towards zero. When applied to narwhal $\textit{(Monodon monoceros)}$ movement data, our analysis suggests that narwhal react to vessels up to 4 kilometres away by decreasing movement persistence and spending more time in deeper waters (average maximum depth of 356m). Overall, we provide a broadly applicable framework for quantifying behavioural responses to stimuli, with applications ranging from determining reaction thresholds to disturbance to estimating the distances at which terrestrial species, such as elephants, detect water.

Figures (9)

• Figure 1: Narwhal tracks, with a 30 minute resolution (yellow, with darker points = deeper) and vessel positions (red) for the first week of August 2017.
• Figure 2: Estimates of ${\beta}_0$ obtained using the lasso-penalized THMM across different sample sizes (a) in the presence of disturbances (scenario 1$.$a) and (b) under the null model (scenario 1$.$b). The red dotted lines correspond to the true value of ${{\beta}}_0$ for different sample sizes. To improve readability, four outliers (estimates exceeding 2) from the sample size of $1,000$ were excluded in (b).
• Figure 3: Estimates of ${\boldsymbol{{\beta}}_0} = ({{\beta}}_0^1,{{\beta}}_0^2)$ obtained using the lasso-penalized THMM in the bivariate setting across different scenarios with sample size $10,000$: (a) both covariates have different disturbance thresholds $\boldsymbol{\beta}_0 = (1.90,1.33)$, (b) only one covariate has an active threshold $\boldsymbol{\beta}_0 = (1.90,0)$, and (c) neither covariate has a threshold $\boldsymbol{{\beta}}_0 = (0,0)$. The red dotted lines correspond to the true value of for each element of ${\boldsymbol{{\beta}}_0}$.
• Figure 4: Estimates from the three-state THMM applied to narwhal movement data. Each colour corresponds to a different state.
• Figure 5: Time series of unstandardized covariates $\{{u}_{1,t}\}_{t=1}^T$ with the thresholds values used in the simulation study: (a) corresponds to no disturbance (scenarios 1$.$b), (b) shows $\{{u}_{1,t}\}_t$ highlighted in red when exceeding the threshold defined in scenario 1$.$a, and panel (c) shows $\{{u}_{1,t}\}_t$ in red when exceeding thresholds defined in scenario 2$.$a.