Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An In-Depth Analysis of Data Reduction Methods for Sustainable Deep Learning

Víctor Toscano-Durán, Javier Perera-Lago, Eduardo Paluzo-Hidalgo, Rocío Gonzalez-Diaz, Miguel Ángel Gutierrez-Naranjo, Matteo Rucco

TL;DR

This paper presents up to eight different methods to reduce the size of a tabular training dataset, and develops a Python package to apply them and introduces a representativeness metric based on topology to measure the similarity between the reduced datasets and the full training dataset.

Abstract

In recent years, Deep Learning has gained popularity for its ability to solve complex classification tasks, increasingly delivering better results thanks to the development of more accurate models, the availability of huge volumes of data and the improved computational capabilities of modern computers. However, these improvements in performance also bring efficiency problems, related to the storage of datasets and models, and to the waste of energy and time involved in both the training and inference processes. In this context, data reduction can help reduce energy consumption when training a deep learning model. In this paper, we present up to eight different methods to reduce the size of a tabular training dataset, and we develop a Python package to apply them. We also introduce a representativeness metric based on topology to measure how similar are the reduced datasets and the full training dataset. Additionally, we develop a methodology to apply these data reduction methods to image datasets for object detection tasks. Finally, we experimentally compare how these data reduction methods affect the representativeness of the reduced dataset, the energy consumption and the predictive performance of the model.

An In-Depth Analysis of Data Reduction Methods for Sustainable Deep Learning

TL;DR

This paper presents up to eight different methods to reduce the size of a tabular training dataset, and develops a Python package to apply them and introduces a representativeness metric based on topology to measure the similarity between the reduced datasets and the full training dataset.

Abstract

In recent years, Deep Learning has gained popularity for its ability to solve complex classification tasks, increasingly delivering better results thanks to the development of more accurate models, the availability of huge volumes of data and the improved computational capabilities of modern computers. However, these improvements in performance also bring efficiency problems, related to the storage of datasets and models, and to the waste of energy and time involved in both the training and inference processes. In this context, data reduction can help reduce energy consumption when training a deep learning model. In this paper, we present up to eight different methods to reduce the size of a tabular training dataset, and we develop a Python package to apply them. We also introduce a representativeness metric based on topology to measure how similar are the reduced datasets and the full training dataset. Additionally, we develop a methodology to apply these data reduction methods to image datasets for object detection tasks. Finally, we experimentally compare how these data reduction methods affect the representativeness of the reduced dataset, the energy consumption and the predictive performance of the model.
Paper Structure (33 sections, 19 equations, 19 figures, 7 tables, 11 algorithms)

This paper contains 33 sections, 19 equations, 19 figures, 7 tables, 11 algorithms.

Table of Contents

  1. Introduction
  2. Key Findings and Contributions
  3. Python Library: Data Reduction Methods
  4. Data Reduction for Images
  5. Summary of the Main Results and Conclusions
  6. Background
  7. Multi-layer Perceptrons
  8. Classification of Tabular Data
  9. Data Reduction
  10. Object Detection from Images
  11. Defining Object Detection
  12. Object Detection with YOLOv5
  13. Metrics
  14. Representativeness Metrics
  15. Performance Metrics
  16. ...and 18 more sections

Figures (19)

  • Figure 1: Architecture of YOLOv5 YOLOV5Arquitectura, including three main parts: backbone, neck and head. The "backbone" is responsible for extracting fundamental features from the image, such as edges and textures. The "neck" is used to extract feature pyramids, which helps the model to generalize well to objects of different sizes and scales. Finally, the "head" is responsible for the final prediction, generating the coordinates and classes of the detected objects.
  • Figure 2: Collision: Reduction + training time
  • Figure 3: Collision: Reduction + training carbon
  • Figure 4: Collision: Reduction + $\varepsilon$-representativeness
  • Figure 5: Collision: Reduction + Accuracy
  • ...and 14 more figures