Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Contrastive Self-Supervised Learning at the Edge: An Energy Perspective

Fernanda Famá, Roberto Pereira, Charalampos Kalalas, Paolo Dini, Lorena Qendro, Fahim Kawsar, Mohammad Malekzadeh

TL;DR

...

Abstract

While contrastive learning (CL) shows considerable promise in self-supervised representation learning, its deployment on resource-constrained devices remains largely underexplored. The substantial computational demands required for training conventional CL frameworks pose a set of challenges, particularly in terms of energy consumption, data availability, and memory usage. We conduct an evaluation of four widely used CL frameworks: SimCLR, MoCo, SimSiam, and Barlow Twins. We focus on the practical feasibility of these CL frameworks for edge and fog deployment, and introduce a systematic benchmarking strategy that includes energy profiling and reduced training data conditions. Our findings reveal that SimCLR, contrary to its perceived computational cost, demonstrates the lowest energy consumption across various data regimes. Finally, we also extend our analysis by evaluating lightweight neural architectures when paired with CL frameworks. Our study aims to provide insights into the resource implications of deploying CL in edge/fog environments with limited processing capabilities and opens several research directions for its future optimization.

Contrastive Self-Supervised Learning at the Edge: An Energy Perspective

TL;DR

...

Abstract

While contrastive learning (CL) shows considerable promise in self-supervised representation learning, its deployment on resource-constrained devices remains largely underexplored. The substantial computational demands required for training conventional CL frameworks pose a set of challenges, particularly in terms of energy consumption, data availability, and memory usage. We conduct an evaluation of four widely used CL frameworks: SimCLR, MoCo, SimSiam, and Barlow Twins. We focus on the practical feasibility of these CL frameworks for edge and fog deployment, and introduce a systematic benchmarking strategy that includes energy profiling and reduced training data conditions. Our findings reveal that SimCLR, contrary to its perceived computational cost, demonstrates the lowest energy consumption across various data regimes. Finally, we also extend our analysis by evaluating lightweight neural architectures when paired with CL frameworks. Our study aims to provide insights into the resource implications of deploying CL in edge/fog environments with limited processing capabilities and opens several research directions for its future optimization.

Paper Structure

This paper contains 17 sections, 3 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Self-Supervised Contrastive Learning
  4. Contrastive Learning at the Edge
  5. Methodology
  6. Measuring Energy Consumption
  7. Measuring Quality of Representation
  8. Numerical Evaluation
  9. Experimental Setup
  10. Data-Performance Trade-off
  11. Energy Profiling
  12. CL Frameworks
  13. Neural Network Architectures
  14. Quality of Representations
  15. Impact of Transformations
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Illustration of the key operating principles for key representative CL frameworks.
  • Figure 2: Energy consumption (dashed lines, left y-axis) and accuracy (solid lines, right y-axis) as the amount of available training data is reduced.
  • Figure 3: Measuring energy consumption components for SimCLR, MoCo, and Barlow Twins frameworks considering several NN architectures while the amount of data is reduced from $100\%$ to $20\%$.
  • Figure 4: Impact of transformations on (a) energy consumption and (b) accuracy performance.