On the Transfer of Collinearity to Computer Vision

Abstract

Collinearity is a visual perception phenomenon in the human brain that amplifies spatially aligned edges arranged along a straight line. However, it is vague for which purpose humans might have this principle in the real-world, and its utilization in computer vision and engineering applications even is a largely unexplored field. In this work, our goal is to transfer the collinearity principle to computer vision, and we explore the potential usages of this novel principle for computer vision applications. We developed a prototype model to exemplify the principle, then tested it systematically, and benchmarked it in the context of four use cases. Our cases are selected to spawn a broad range of potential applications and scenarios: sketching the combination of collinearity with deep learning (case I and II), using collinearity with saliency models (case II), and as a feature detector (case I). In the first use case, we found that collinearity is able to improve the fault detection of wafers and obtain a performance increase by a factor 1.24 via collinearity (decrease of the error rate from 6.5% to 5.26%). In the second use case, we test the defect recognition in nanotechnology materials and achieve a performance increase by 3.2x via collinearity (deep learning, error from 21.65% to 6.64%), and also explore saliency models. As third experiment, we cover occlusions; while as fourth experiment, we test ImageNet and observe that it might not be very beneficial for ImageNet. Therefore, we can assemble a list of scenarios for which collinearity is beneficial (wafers, nanotechnology, occlusions), and for what is not beneficial (ImageNet). Hence, we infer collinearity might be suitable for industry applications as it helps if the image structures of interest are man-made because they often consist of lines. Our work provides another tool for CV, hope to capture the power of human processing.

Figures (21)

• Figure 1: a) Illustration of the collinearity principle. b) Transfer from psychology to the computer vision domain. b, left) Psychology, reprinted from Maniglia2015a. b, right) Computer vision domain: Fault detection in semiconductor wafers Beuth2021. Defect recognition in nanotechnology materials Carrera2017, and ImageNet subset taken from Krizhevsky2017 (in clock-wise direction). The green circles denote the structures of interests in Beuth2021Carrera2017.
• Figure 2: Neural model of collinearity.
• Figure 3: Neural connections which mediate the concept of collinearity in the model. The connections link neurons along a virtual line (collinearity), and their influence enhances the central neuron. They are located inside the collinearity layer. Note, only one of the multiple connection orientations is shown. In the right inset, it is shown the assumed location of the model in the human brain (primary visual cortex, V1) and the cortical layers (reprinted from Brodmann1909).
• Figure 4: Precise spatial extent as well as width of the collinearity connectivity. The receiving neuron is marked in gray, and a red color denotes that a neuron at this position has a connection to the receiving neuron, i.e. the color denotes the weight strength. Both dimensions are measured in $\lambda$, with 1 $\lambda$ = size of wavelength of the Gabor, e.g. $1\,\lambda\,\hat{=}\,5\,$pixels. The connections in the direction of the spatial extent have a minimum distance, i.e. there is a region without a collinearity influence around the neuron, and the connections stretch until a maximum distance. Note, only the connectivity matrix for a single orientation is shown, each orientation has its own connectivity pattern, which is rotated appropriately for each orientation. See main text for details (Sec. \ref{['sec:modelNeuroAndKernel']} and matrix $w_{x,x',l}$ in Eq. \ref{['eq:col3']}).
• Figure 5: Basic example of operation of the collinearity principle. (a) If line elements are arranged along a longer line pattern, they are enhanced. The outer line elements are denoted as flankers. (b) If the outer flankers are not aligned, i.e. by $90^\circ$ rotated, they are not enhanced. On the left are shown the image setups, whereas on the right are shown the neural responses, which are read out for the central position of the collinearity layer (marked by the arrow, response $r\Area{Col}$, cf. Sec. \ref{['sec:eq']}).