Multi-objective hybrid knowledge distillation for efficient deep learning in smart agriculture

TL;DR

The paper tackles edge deployment in smart agriculture by introducing a multi-objective hybrid knowledge distillation framework that transfers knowledge from a ResNet18 teacher to a compact student built from inverted residuals and DenseNet-style connectivity. It trains online with four complementary losses—hard-label, feature-based, response-based, and self-distillation—facilitated by an auxiliary branch for intermediate supervision. Across five agricultural datasets, the student achieves near-teacher accuracy at a fraction of the compute and parameter cost, and demonstrates robust generalization and interpretable Grad-CAM attention patterns. This approach enables practical, on-device deployment for rice seed identification and plant leaf disease classification while maintaining high predictive performance.

Abstract

Deploying deep learning models on resource-constrained edge devices remains a major challenge in smart agriculture due to the trade-off between computational efficiency and recognition accuracy. To address this challenge, this study proposes a hybrid knowledge distillation framework for developing a lightweight yet high-performance convolutional neural network. The proposed approach designs a customized student model that combines inverted residual blocks with dense connectivity and trains it under the guidance of a ResNet18 teacher network using a multi-objective strategy that integrates hard-label supervision, feature-level distillation, response-level distillation, and self-distillation. Experiments are conducted on a rice seed variety identification dataset containing nine varieties and further extended to four plant leaf disease datasets, including rice, potato, coffee, and corn, to evaluate generalization capability. On the rice seed variety classification task, the distilled student model achieves an accuracy of 98.56%, which is only 0.09% lower than the teacher model (98.65%), while requiring only 0.68 GFLOPs and approximately 1.07 million parameters. This corresponds to a reduction of about 2.7 times in computational cost and more than 10 times in model size compared with the ResNet18 teacher model. In addition, compared with representative pretrained models, the proposed student reduces the number of parameters by more than 6 times relative to DenseNet121 and by over 80 times compared with the Vision Transformer (ViT) architecture, while maintaining comparable or superior classification accuracy. Consistent performance gains across multiple plant leaf disease datasets further demonstrate the robustness, efficiency, and strong deployment potential of the proposed framework for hardware-limited smart agriculture systems.

Figures (13)

• Figure 1: Overall framework of the proposed knowledge distillation approach
• Figure 2: Architecture of a block in the student model.
• Figure 3: Design of the auxiliary branches used in the teacher and student models
• Figure 4: Sample images from the rice variety dataset containing 9 different rice varieties.
• Figure 5: Sample images from the rice leaf disease dataset showing various leaf diseases.