A Comparative Study of Open Source Computer Vision Models for Application on Small Data: The Case of CFRP Tape Laying

TL;DR

The results show that the amount of data required to successfully train an AI model can be drastically reduced, and the use of smaller models does not necessarily lead to a loss of performance.

Abstract

In the realm of industrial manufacturing, Artificial Intelligence (AI) is playing an increasing role, from automating existing processes to aiding in the development of new materials and techniques. However, a significant challenge arises in smaller, experimental processes characterized by limited training data availability, questioning the possibility to train AI models in such small data contexts. In this work, we explore the potential of Transfer Learning to address this challenge, specifically investigating the minimum amount of data required to develop a functional AI model. For this purpose, we consider the use case of quality control of Carbon Fiber Reinforced Polymer (CFRP) tape laying in aerospace manufacturing using optical sensors. We investigate the behavior of different open-source computer vision models with a continuous reduction of the training data. Our results show that the amount of data required to successfully train an AI model can be drastically reduced, and the use of smaller models does not necessarily lead to a loss of performance.

Figures (4)

• Figure 1: Automated Head for CFRP tape laying with optical sensor for quality control dlr137804.
• Figure 2: An exemplary high-resolution 16-bit TIFF image with annotation showing the measured height profiles at high contrast. The middle of the pictures shows the actual track consisting of three tapes arranged next to each other, displayed in light gray. To the left is the previous track, to the right the flat dark gray background. Each tape is 12,54 mm wide and may reach several meters in length, depending on the layup. The upper section of the image shows the beginning of the laid tape. On the left side in the previous track between the middle and right tape there is a gap (highlighted in blue), and at the border between the previous and the actual track there is an overlap (highlighted in red). Each of the 65,536 grayscale values of the original TIFF corresponds to a height differential of 1.77 micrometers, i.e., to represent the height difference of a tape, 79 grayscale values are required dlr137804.
• Figure 3: Analysis of sensitivity to training dataset size for fine-tuned models from table \ref{['tab:models']}: Averaged $\text{F}_1$ score versus number of examples per category in the training dataset. The results are colored in relation to the PyTorch model binary file sizes in a continuous color gradient from blue (small) to red (large).
• Figure 4: Analysis of performance relative to model size: Averaged $\text{F}_1$ scores versus PyTorch model binary file sizes for fine-tuned models from Table \ref{['tab:models']} across four training datasets with diverse category sample sizes.