A Multitask Deep Learning Model for Classification and Regression of Hyperspectral Images: Application to the large-scale dataset

TL;DR

This work presents a multitask deep-learning framework for simultaneous classification and regression on hyperspectral images, validated on the TAIGA dataset with 13 forest variables. The model employs a shared encoder (ResNet with Dense ASPP), a spectral-spatial residual attention block, and task-specific decoders, augmented by a robust multitask loss that incorporates cost-sensitive/focal losses and uncertainty or GradNorm-based balancing. Empirical results show the proposed approach achieves state-of-the-art performance (OA ~95.92%, RMSE ~0.027) across both categorical and continuous targets, with strong robustness across 10 seeds and favorable computational efficiency. The study demonstrates the benefits of shared representation learning for diverse HSI tasks and provides a reproducible framework for large-scale hyperspectral multitask analysis.

Abstract

Multitask learning is a widely recognized technique in the field of computer vision and deep learning domain. However, it is still a research question in remote sensing, particularly for hyperspectral imaging. Moreover, most of the research in the remote sensing domain focuses on small and single-task-based annotated datasets, which limits the generalizability and scalability of the developed models to more diverse and complex real-world scenarios. Thus, in this study, we propose a multitask deep learning model designed to perform multiple classification and regression tasks simultaneously on hyperspectral images. We validated our approach on a large hyperspectral dataset called TAIGA, which contains 13 forest variables, including three categorical variables and ten continuous variables with different biophysical parameters. We design a sharing encoder and task-specific decoder network to streamline feature learning while allowing each task-specific decoder to focus on the unique aspects of its respective task. Additionally, a dense atrous pyramid pooling layer and attention network were integrated to extract multi-scale contextual information and enable selective information processing by prioritizing task-specific features. Further, we computed multitask loss and optimized its parameters for the proposed framework to improve the model performance and efficiency across diverse tasks. A comprehensive qualitative and quantitative analysis of the results shows that the proposed method significantly outperforms other state-of-the-art methods. We trained our model across 10 seeds/trials to ensure robustness. Our proposed model demonstrates higher mean performance while maintaining lower or equivalent variability. To make the work reproducible, the codes will be available at https://github.com/Koushikey4596/Multitask-Deep-Learning-Model-for-Taiga-datatset.

Figures (14)

• Figure 1: Framework of the proposed multitask deep learning model for classification and regression of hyperspectral image. Each output has its own loss criterion, Cross-Entropy (CE) for classification, and Mean Absolute Error (MAE) for regression.
• Figure 2: Visualization of classification labels (a-c) and class distribution (d-f) for different categorical variables, namely, fertility class, soil type, and main tree species.
• Figure 3: Visualization of normalized regression labels from 0-1 for all ten continuous variables, viz., a) basal area b)mean dbh c)stem density d)mean height e)% of pine f) % of spruce g) % of birch h)woody biomass i)leaf area index j) effective leaf area index
• Figure 4: Visualization of ROI separability to know the class overlap and correlation between different forest variables (continuous and categorical)
• Figure 5: The split dataset into training and test sets such that no two samples overlap each other and not a single pixel should present in both training and testing sets.