Bayesian Modelling of Multi-Year Crop Type Classification Using Deep Neural Networks and Hidden Markov Models

TL;DR

Presents a cascade Bayesian framework for multiyear crop-type classification in Satellite Image Time Series (SITS) by coupling a Transformer Encoder emission model with an HMM transition layer. The approach estimates $p(\mathbf{x}^{(y)}|\omega^{(y)})$ with the TE and enforces $P(\omega^{(y+1)}|\omega^{(y)})$ via a custom HMM layer, enabling forward-backward cascade inference over six years. End-to-end fine-tuning on a six-year Sentinel-2 Austria dataset with 47 crop types yields the best mean F1 of $mF1 \approx 73.59\%$ for a 1st-order cascade, outperforming year-wise DNN baselines. The results demonstrate the value of temporal consistency in SITS crop mapping and offer a scalable framework for multiyear LULC applications, with future directions in change detection and weakly supervised learning.

Abstract

The temporal consistency of yearly land-cover maps is of great importance to model the evolution and change of the land cover over the years. In this paper, we focus the attention on a novel approach to classification of yearly satellite image time series (SITS) that combines deep learning with Bayesian modelling, using Hidden Markov Models (HMMs) integrated with Transformer Encoder (TE) based DNNs. The proposed approach aims to capture both i) intricate temporal correlations in yearly SITS and ii) specific patterns in multiyear crop type sequences. It leverages the cascade classification of an HMM layer built on top of the TE, discerning consistent yearly crop-type sequences. Validation on a multiyear crop type classification dataset spanning 47 crop types and six years of Sentinel-2 acquisitions demonstrates the importance of modelling temporal consistency in the predicted labels. HMMs enhance the overall performance and F1 scores, emphasising the effectiveness of the proposed approach.

Figures (1)

• Figure 1: Flowchart of the proposed cascade-based classification approach, combining a DNN model with an HMM layer.