Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

InFusionLayer: a CFA-based ensemble tool to generate new classifiers for learning and modeling

Eric Roginek, Jingyan Xu, D. Frank. Hsu

Abstract

Ensemble learning is a well established body of methods for machine learning to enhance predictive performance by combining multiple algorithms/models. Combinatorial Fusion Analysis (CFA) has provided method and practice for combining multiple scoring systems, using rank-score characteristic (RSC) function and cognitive diversity (CD), including ensemble method and model fusion. However, there is no general-purpose Python tool available that incorporate these techniques. In this paper we introduce \texttt{InFusionLayer}, a machine learning architecture inspired by CFA at the system fusion level that uses a moderate set of base models to optimize unsupervised and supervised learning multiclassification problems. We demonstrate \texttt{InFusionLayer}'s ease of use for PyTorch, TensorFlow, and Scikit-learn workflows by validating its performance on various computer vision datasets. Our results highlight the practical advantages of incorporating distinctive features of RSC function and CD, paving the way for more sophisticated ensemble learning applications in machine learning. We open-sourced our code to encourage continuing development and community accessibility to leverage CFA on github: https://github.com/ewroginek/Infusion

InFusionLayer: a CFA-based ensemble tool to generate new classifiers for learning and modeling

Abstract

Ensemble learning is a well established body of methods for machine learning to enhance predictive performance by combining multiple algorithms/models. Combinatorial Fusion Analysis (CFA) has provided method and practice for combining multiple scoring systems, using rank-score characteristic (RSC) function and cognitive diversity (CD), including ensemble method and model fusion. However, there is no general-purpose Python tool available that incorporate these techniques. In this paper we introduce \texttt{InFusionLayer}, a machine learning architecture inspired by CFA at the system fusion level that uses a moderate set of base models to optimize unsupervised and supervised learning multiclassification problems. We demonstrate \texttt{InFusionLayer}'s ease of use for PyTorch, TensorFlow, and Scikit-learn workflows by validating its performance on various computer vision datasets. Our results highlight the practical advantages of incorporating distinctive features of RSC function and CD, paving the way for more sophisticated ensemble learning applications in machine learning. We open-sourced our code to encourage continuing development and community accessibility to leverage CFA on github: https://github.com/ewroginek/Infusion
Paper Structure (20 sections, 17 equations, 4 figures, 3 tables, 2 algorithms)

This paper contains 20 sections, 17 equations, 4 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Previous work
  3. Combinatoriaral Fusion Analysis
  4. Multiclassification
  5. Methodology
  6. Experiments
  7. Dataset
  8. InFusionLayer
  9. Initialization and Setup
  10. Experimental Setup
  11. Model Combinations
  12. Combinatorial Fusion Analysis
  13. Model Fusion
  14. Batch Processing
  15. Accuracy Calculation
  16. ...and 5 more sections

Figures (4)

  • Figure 1: InFusionLayer workflow for a single forward pass. A set of independent models $A, B, C, D,$ and $E$ are pretrained on the same dataset. Each model is then assigned a prediction task for a given test set. Model score predictions are used as input nodes for InFusionLayer. CFA obtains a derived rank function and performs three weighted combinations on model score predictions (green) and rank predictions (red). Top-k score models and rank models are selected within-groups prior to a final top-k comparison between groups. These models are saved and the top performing fusion model is selected as output.
  • Figure 2: Average Combination: ASC (blue) and ARC (red)
  • Figure 3: Weighted Combination by Diversity Strength: WSCDS (blue) and WRCDS (red)
  • Figure 4: Weighted Combination by Performance: WSCP (blue) and WRCP (red)