Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semi-adaptive Synergetic Two-way Pseudoinverse Learning System

Binghong Liu, Ziqi Zhao, Shupan Li, Ke Wang

TL;DR

This paper addresses the inefficiencies and hyperparameter sensitivity of gradient-descent training by introducing a semi-adaptive, non-gradient two-way pseudoinverse learning system built on a synergetic learning framework. Each elementary model combines forward and backward PIL-based learning with feature fusion, and its depth grows automatically in a data-driven manner, enabling parallelizable training. Empirical results across MNIST, Fashion-MNIST, NORB, and various UCI/OpenML datasets show superior accuracy relative to non-gradient baselines and competitive efficiency versus gradient-based networks. The approach reduces manual architecture tuning and offers scalable, distributed deployment potential for faster, more robust learning.

Abstract

Deep learning has become a crucial technology for making breakthroughs in many fields. Nevertheless, it still faces two important challenges in theoretical and applied aspects. The first lies in the shortcomings of gradient descent based learning schemes which are time-consuming and difficult to determine the learning control hyperparameters. Next, the architectural design of the model is usually tricky. In this paper, we propose a semi-adaptive synergetic two-way pseudoinverse learning system, wherein each subsystem encompasses forward learning, backward learning, and feature concatenation modules. The whole system is trained using a non-gradient descent learning algorithm. It simplifies the hyperparameter tuning while improving the training efficiency. The architecture of the subsystems is designed using a data-driven approach that enables automated determination of the depth of the subsystems. We compare our method with the baselines of mainstream non-gradient descent based methods and the results demonstrate the effectiveness of our proposed method. The source code for this paper is available at http://github.com/B-berrypie/Semi-adaptive-Synergetic-Two-way-Pseudoinverse-Learning-System}{http://github.com/B-berrypie/Semi-adaptive-Synergetic-Two-way-Pseudoinverse-Learning-System.

Semi-adaptive Synergetic Two-way Pseudoinverse Learning System

TL;DR

This paper addresses the inefficiencies and hyperparameter sensitivity of gradient-descent training by introducing a semi-adaptive, non-gradient two-way pseudoinverse learning system built on a synergetic learning framework. Each elementary model combines forward and backward PIL-based learning with feature fusion, and its depth grows automatically in a data-driven manner, enabling parallelizable training. Empirical results across MNIST, Fashion-MNIST, NORB, and various UCI/OpenML datasets show superior accuracy relative to non-gradient baselines and competitive efficiency versus gradient-based networks. The approach reduces manual architecture tuning and offers scalable, distributed deployment potential for faster, more robust learning.

Abstract

Deep learning has become a crucial technology for making breakthroughs in many fields. Nevertheless, it still faces two important challenges in theoretical and applied aspects. The first lies in the shortcomings of gradient descent based learning schemes which are time-consuming and difficult to determine the learning control hyperparameters. Next, the architectural design of the model is usually tricky. In this paper, we propose a semi-adaptive synergetic two-way pseudoinverse learning system, wherein each subsystem encompasses forward learning, backward learning, and feature concatenation modules. The whole system is trained using a non-gradient descent learning algorithm. It simplifies the hyperparameter tuning while improving the training efficiency. The architecture of the subsystems is designed using a data-driven approach that enables automated determination of the depth of the subsystems. We compare our method with the baselines of mainstream non-gradient descent based methods and the results demonstrate the effectiveness of our proposed method. The source code for this paper is available at http://github.com/B-berrypie/Semi-adaptive-Synergetic-Two-way-Pseudoinverse-Learning-System}{http://github.com/B-berrypie/Semi-adaptive-Synergetic-Two-way-Pseudoinverse-Learning-System.

Paper Structure

This paper contains 18 sections, 23 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Pseudoinverse Learning based Autoencoder
  4. Synergetic Learning System
  5. Methodology
  6. Elementary Model
  7. Forward Learning
  8. Backward Learning
  9. Feature Fusion
  10. Synergetic System
  11. Training Strategy
  12. Parallelizability
  13. Early Stopping
  14. Experiments
  15. Data Sets
  16. ...and 3 more sections

Figures (6)

  • Figure 1: A schematic of the methodology to build stacked PILAE
  • Figure 2: A diagram of the methodology to build SLS
  • Figure 3: The structure of semi-adaptive synergetic two-way pseudoinverse learning system
  • Figure 4: The structure of the elementary model
  • Figure 5: The accuracy ranking box plots of our method and baselines
  • ...and 1 more figures