Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transformer based Multi-task Fusion Network for Food Spoilage Detection and Shelf life Forecasting

Mounika Kanulla, Rajasree Dadigi, Sailaja Thota, Vivek Yelleti

TL;DR

The paper tackles automatic vegetable quality assessment by introducing multi-task fusion networks that perform vegetable type classification, spoilage detection, and shelf-life forecasting. It compares CNN-only, transformer, and fusion architectures, finding that CNN+DeiT Transformer fusion yields the strongest overall performance, with high vegetable classification accuracy and robust shelf-life prediction, even under noisy conditions. The work emphasizes robustness to image quality variations and provides interpretability through LIME explanations, culminating in an end-to-end Flask-based deployment. Overall, multi-task fusion approaches prove more effective and practical for real-world food quality monitoring than single-task models.

Abstract

Food wastage is one of the critical challenges in the agricultural supply chain, and accurate and effective spoilage detection can help to reduce it. Further, it is highly important to forecast the spoilage information. This aids the longevity of the supply chain management in the agriculture field. This motivated us to propose fusion based architectures by combining CNN with LSTM and DeiT transformer for the following multi-tasks simultaneously: (i) vegetable classification, (ii) food spoilage detection, and (iii) shelf life forecasting. We developed a dataset by capturing images of vegetables from their fresh state until they were completely spoiled. From the experimental analysis it is concluded that the proposed fusion architectures CNN+CNN-LSTM and CNN+DeiT Transformer outperformed several deep learning models such as CNN, VGG16, ResNet50, Capsule Networks, and DeiT Transformers. Overall, CNN + DeiT Transformer yielded F1-score of 0.98 and 0.61 in vegetable classification and spoilage detection respectively and mean squared error (MSE) and symmetric mean absolute percentage error (SMAPE) of 3.58, and 41.66% respectively in spoilage forecasting. Further, the reliability of the fusion models was validated on noisy images and integrated with LIME to visualize the model decisions.

Transformer based Multi-task Fusion Network for Food Spoilage Detection and Shelf life Forecasting

TL;DR

The paper tackles automatic vegetable quality assessment by introducing multi-task fusion networks that perform vegetable type classification, spoilage detection, and shelf-life forecasting. It compares CNN-only, transformer, and fusion architectures, finding that CNN+DeiT Transformer fusion yields the strongest overall performance, with high vegetable classification accuracy and robust shelf-life prediction, even under noisy conditions. The work emphasizes robustness to image quality variations and provides interpretability through LIME explanations, culminating in an end-to-end Flask-based deployment. Overall, multi-task fusion approaches prove more effective and practical for real-world food quality monitoring than single-task models.

Abstract

Food wastage is one of the critical challenges in the agricultural supply chain, and accurate and effective spoilage detection can help to reduce it. Further, it is highly important to forecast the spoilage information. This aids the longevity of the supply chain management in the agriculture field. This motivated us to propose fusion based architectures by combining CNN with LSTM and DeiT transformer for the following multi-tasks simultaneously: (i) vegetable classification, (ii) food spoilage detection, and (iii) shelf life forecasting. We developed a dataset by capturing images of vegetables from their fresh state until they were completely spoiled. From the experimental analysis it is concluded that the proposed fusion architectures CNN+CNN-LSTM and CNN+DeiT Transformer outperformed several deep learning models such as CNN, VGG16, ResNet50, Capsule Networks, and DeiT Transformers. Overall, CNN + DeiT Transformer yielded F1-score of 0.98 and 0.61 in vegetable classification and spoilage detection respectively and mean squared error (MSE) and symmetric mean absolute percentage error (SMAPE) of 3.58, and 41.66% respectively in spoilage forecasting. Further, the reliability of the fusion models was validated on noisy images and integrated with LIME to visualize the model decisions.
Paper Structure (18 sections, 7 figures, 5 tables)

This paper contains 18 sections, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Problem statement
  3. Novelty Claims
  4. Literature Review
  5. Proposed Methodology
  6. Proposed Model -- 1: MobileNetV2 and DeiT Transformer Fusion Model
  7. Proposed Model -- 2: MobileNetV2 and MobileNetV2 -- LSTM Fusion
  8. Explainability for the proposed Model
  9. Dataset Description
  10. Implementation
  11. Classification and Fusion Models
  12. Workflow
  13. Model Workflow
  14. Results And Discussions
  15. Performance of models on the original dataset
  16. ...and 3 more sections

Figures (7)

  • Figure 1: Proposed Methodology-1: MobileNetV2 + DeiT tranformer
  • Figure 2: Proposed Methodology-2: MobileNetv2 + MobileNetv2_LSTM fusion
  • Figure 3: LIME--based explainability maps highlighting the regions influencing vegetable class prediction, spoilage detection, and day-wise freshness regression in the proposed model 1.
  • Figure 4: LIME--based explainability maps highlighting the regions influencing vegetable class prediction, spoilage detection, and day-wise freshness regression in the proposed model 2.
  • Figure 5: Sample images from the self-constructed dataset illustrating fresh and spoiled conditions across different fruits and vegetables.
  • ...and 2 more figures