Efficient first-order algorithms for large-scale, non-smooth maximum entropy models with application to wildfire science

TL;DR

This paper presents novel optimization algorithms that overcome the shortcomings of state-of-the-art algorithms for training large-scale, non-smooth MaxEnt models and considers the problem of estimating probabilities of fire occurrences as a function of ecological features in the Western US MTBS-Interagency wildfire data set.

Abstract

Maximum entropy (Maxent) models are a class of statistical models that use the maximum entropy principle to estimate probability distributions from data. Due to the size of modern data sets, Maxent models need efficient optimization algorithms to scale well for big data applications. State-of-the-art algorithms for Maxent models, however, were not originally designed to handle big data sets; these algorithms either rely on technical devices that may yield unreliable numerical results, scale poorly, or require smoothness assumptions that many practical Maxent models lack. In this paper, we present novel optimization algorithms that overcome the shortcomings of state-of-the-art algorithms for training large-scale, non-smooth Maxent models. Our proposed first-order algorithms leverage the Kullback-Leibler divergence to train large-scale and non-smooth Maxent models efficiently. For Maxent models with discrete probability distribution of $n$ elements built from samples, each containing $m$ features, the stepsize parameters estimation and iterations in our algorithms scale on the order of $O(mn)$ operations and can be trivially parallelized. Moreover, the strong $\ell_{1}$ convexity of the Kullback--Leibler divergence allows for larger stepsize parameters, thereby speeding up the convergence rate of our algorithms. To illustrate the efficiency of our novel algorithms, we consider the problem of estimating probabilities of fire occurrences as a function of ecological features in the Western US MTBS-Interagency wildfire data set. Our numerical results show that our algorithms outperform the state of the arts by one order of magnitude and yield results that agree with physical models of wildfire occurrence and previous statistical analyses of wildfire drivers.

Figures (4)

• Figure 1: Wildfire activity in the western United States from 1984 to 2020. (Left) Fire locations of all fires (black dots) in the Western US MTBS-Interagency (WUMI) data set; also shown are three ecological divisions characterized by their primary vegetation type – forests (green), deserts (yellow), and plains (gray). (Right) Prior distribution indicating mean fire probability across all calendar months.
• Figure 2: Spatial probability plot for different hyperparameter values with elastic net penalty parameter $\alpha = \{0.95, 0.40, 0.15, 0.05 \}$.
• Figure 3: Same as \ref{['fig:hypvar_elastic_net']} but for (left) the non-overlapping group lasso with $\alpha = 1$, and (right) the $l_\infty$ Maxent models respectively.
• Figure 4: Number of non-zero coefficients along the regularization path plots for elastic net penalty parameter $\alpha = \{0.95, 0.40, 0.15, 0.05 \}$. The dashed vertical lines highlight the $t/t_{\rm max}$ value at which the first feature of the group indicated by inset text is selected.