Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LightFF: Lightweight Inference for Forward-Forward Algorithm

Amin Aminifar, Baichuan Huang, Azra Abtahi, Amir Aminifar

TL;DR

This work tackles the energy cost of inference in forward-only neural networks trained with Forward-Forward variants. It introduces LightFF, a lightweight inference framework that enables early exits via layer-wise confidence using per-layer goodness and softmax-based assessments, delivering two procedures: multi-pass and one-pass. Empirical results across MNIST, CIFAR-10, and wearable-health datasets (epilepsy and arrhythmia) show LightFF achieves comparable accuracy to baseline forward-only methods while substantially reducing the number of processed layers, MAC operations, and execution time. The approach holds practical significance for real-time, resource-constrained applications, including perpetual monitoring on wearables, and contributes to the broader goal of energy-efficient AI with forward-only learning paradigms.

Abstract

The human brain performs tasks with an outstanding energy efficiency, i.e., with approximately 20 Watts. The state-of-the-art Artificial/Deep Neural Networks (ANN/DNN), on the other hand, have recently been shown to consume massive amounts of energy. The training of these ANNs/DNNs is done almost exclusively based on the back-propagation algorithm, which is known to be biologically implausible. This has led to a new generation of forward-only techniques, including the Forward-Forward algorithm. In this paper, we propose a lightweight inference scheme specifically designed for DNNs trained using the Forward-Forward algorithm. We have evaluated our proposed lightweight inference scheme in the case of the MNIST and CIFAR datasets, as well as two real-world applications, namely, epileptic seizure detection and cardiac arrhythmia classification using wearable technologies, where complexity overheads/energy consumption is a major constraint, and demonstrate its relevance. Our code is available at https://github.com/AminAminifar/LightFF.

LightFF: Lightweight Inference for Forward-Forward Algorithm

TL;DR

This work tackles the energy cost of inference in forward-only neural networks trained with Forward-Forward variants. It introduces LightFF, a lightweight inference framework that enables early exits via layer-wise confidence using per-layer goodness and softmax-based assessments, delivering two procedures: multi-pass and one-pass. Empirical results across MNIST, CIFAR-10, and wearable-health datasets (epilepsy and arrhythmia) show LightFF achieves comparable accuracy to baseline forward-only methods while substantially reducing the number of processed layers, MAC operations, and execution time. The approach holds practical significance for real-time, resource-constrained applications, including perpetual monitoring on wearables, and contributes to the broader goal of energy-efficient AI with forward-only learning paradigms.

Abstract

The human brain performs tasks with an outstanding energy efficiency, i.e., with approximately 20 Watts. The state-of-the-art Artificial/Deep Neural Networks (ANN/DNN), on the other hand, have recently been shown to consume massive amounts of energy. The training of these ANNs/DNNs is done almost exclusively based on the back-propagation algorithm, which is known to be biologically implausible. This has led to a new generation of forward-only techniques, including the Forward-Forward algorithm. In this paper, we propose a lightweight inference scheme specifically designed for DNNs trained using the Forward-Forward algorithm. We have evaluated our proposed lightweight inference scheme in the case of the MNIST and CIFAR datasets, as well as two real-world applications, namely, epileptic seizure detection and cardiac arrhythmia classification using wearable technologies, where complexity overheads/energy consumption is a major constraint, and demonstrate its relevance. Our code is available at https://github.com/AminAminifar/LightFF.
Paper Structure (15 sections, 2 equations, 5 figures, 4 tables)

This paper contains 15 sections, 2 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Lightweight Inference
  3. Inference Based on Multi-Pass Procedure
  4. Inference Based on One-Pass Procedure
  5. Evaluation
  6. Experimental Setup
  7. Datasets
  8. Baselines
  9. Implementation Details
  10. Results
  11. Performance Evaluation
  12. Complexity Evaluation
  13. Detailed Analysis and Discussion
  14. Related Work
  15. Conclusions

Figures (5)

  • Figure 1: Lightweight Procedures for Inference
  • Figure 2: The distribution of MNIST validation data (Green: Negative data; Blue: Positive data) and the corresponding mean values (red vertical lines) in the Forward-Forward algorithm. The distance between the mean values of the negative and positive samples/distributions increases as we consider more layers.
  • Figure 3: The mean values of negative data and positive data in the Forward-Forward algorithm (MP).
  • Figure 4: The average/mean number of layers used by our lightweight inference schemes versus confidence threshold.
  • Figure 5: Probability of the number of layers used, for 2--6 layer networks, four datasets, and three algorithms (MP, OP, and PT). Blue: MP; Green: OP; Yellow: PT. The lightest color represents 1 layer and the darkest color represents 6 layers.