Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning using granularity statistical invariants for classification

Ting-Ting Zhu, Yuan-Hai Shao, Chun-Na Li, Tian Liu

TL;DR

The paper tackles the computational bottleneck of invariant matrices in LUSI for large-scale classification and introduces LUGSI, which employs granularity through clustering to create multiple smaller invariants while combining strong and weak convergence to minimize expected risk. By partitioning data into granules and constructing per-granule invariants via a $\mathbf{v}$ vector, LUGSI converts a large matrix into smaller ones and emphasizes local structure, leading to faster training and improved generalization. The framework extends to nonlinear settings with kernel forms and a DCT-based variant (DCTLUGSI) using a CRO kernel for efficiency, and it theoretically links LUGSI to VSVM and LSSVM as special cases. Empirically, LUGSI achieves superior or competitive accuracy on UCI and NDC datasets, with substantial speedups on large-scale data, validating the benefits of integrating granular structural information into invariant-based learning.

Abstract

Learning using statistical invariants (LUSI) is a new learning paradigm, which adopts weak convergence mechanism, and can be applied to a wider range of classification problems. However, the computation cost of invariant matrices in LUSI is high for large-scale datasets during training. To settle this issue, this paper introduces a granularity statistical invariant for LUSI, and develops a new learning paradigm called learning using granularity statistical invariants (LUGSI). LUGSI employs both strong and weak convergence mechanisms, taking a perspective of minimizing expected risk. As far as we know, it is the first time to construct granularity statistical invariants. Compared to LUSI, the introduction of this new statistical invariant brings two advantages. Firstly, it enhances the structural information of the data. Secondly, LUGSI transforms a large invariant matrix into a smaller one by maximizing the distance between classes, achieving feasibility for large-scale datasets classification problems and significantly enhancing the training speed of model operations. Experimental results indicate that LUGSI not only exhibits improved generalization capabilities but also demonstrates faster training speed, particularly for large-scale datasets.

Learning using granularity statistical invariants for classification

TL;DR

The paper tackles the computational bottleneck of invariant matrices in LUSI for large-scale classification and introduces LUGSI, which employs granularity through clustering to create multiple smaller invariants while combining strong and weak convergence to minimize expected risk. By partitioning data into granules and constructing per-granule invariants via a vector, LUGSI converts a large matrix into smaller ones and emphasizes local structure, leading to faster training and improved generalization. The framework extends to nonlinear settings with kernel forms and a DCT-based variant (DCTLUGSI) using a CRO kernel for efficiency, and it theoretically links LUGSI to VSVM and LSSVM as special cases. Empirically, LUGSI achieves superior or competitive accuracy on UCI and NDC datasets, with substantial speedups on large-scale data, validating the benefits of integrating granular structural information into invariant-based learning.

Abstract

Learning using statistical invariants (LUSI) is a new learning paradigm, which adopts weak convergence mechanism, and can be applied to a wider range of classification problems. However, the computation cost of invariant matrices in LUSI is high for large-scale datasets during training. To settle this issue, this paper introduces a granularity statistical invariant for LUSI, and develops a new learning paradigm called learning using granularity statistical invariants (LUGSI). LUGSI employs both strong and weak convergence mechanisms, taking a perspective of minimizing expected risk. As far as we know, it is the first time to construct granularity statistical invariants. Compared to LUSI, the introduction of this new statistical invariant brings two advantages. Firstly, it enhances the structural information of the data. Secondly, LUGSI transforms a large invariant matrix into a smaller one by maximizing the distance between classes, achieving feasibility for large-scale datasets classification problems and significantly enhancing the training speed of model operations. Experimental results indicate that LUGSI not only exhibits improved generalization capabilities but also demonstrates faster training speed, particularly for large-scale datasets.
Paper Structure (13 sections, 2 theorems, 37 equations, 4 figures, 4 tables)

This paper contains 13 sections, 2 theorems, 37 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Direct methods of estimation of conditional probability function
  3. Learning using granularity statistical invariants (LUGSI)
  4. Granularization
  5. Learning model and solution
  6. Nonlinear LUGSI
  7. Relationship with LUGSI, VSVM and LSSVM
  8. Model complexity analysis
  9. Experiments
  10. Comparison of accuracy on the benchmark datasets
  11. The impact of clustering and on accuracy and training time
  12. The influence of global and local information on the accuracy and training time of NDC datasets with different scales
  13. Conclusion

Key Result

Proposition 1

When each point is treated as an individual class, the LUGSI model degenerates into the LSSVM model. Additionally, if we consider the $V$-matrix as an identity matrix, the VSVM model degenerates into LSSVM.

Figures (4)

  • Figure 1: The impact of clustering in linear space on accuracy and training time on small datasets from UCI.
  • Figure 2: The impact of clustering in nonlinear space on accuracy and training time on small datasets from UCI.
  • Figure 3: The impact of clustering on accuracy and training time of large-scale datasets.
  • Figure 4: The effect of clustering on the accuracy and training time on NDC datasets of different sizes.

Theorems & Definitions (5)

  • Definition 1
  • Proposition 1
  • proof
  • Proposition 2
  • proof