On The Potential of The Fractal Geometry and The CNNs Ability to Encode it

TL;DR

This work investigates whether fractal geometry can be encoded by deep convolutional networks and whether fractal-based features can complement or outperform deep models on structure-centric tasks. By extracting fractal features (ZFrac) across multiple scales and comparing them to DL representations using CCA and CKA, the authors show that standard CNNs do not encode fractal geometry in any layer. A shallow network trained on ZFrac features (ZFrac+NN) achieves comparable or superior accuracy to DL models on several datasets while requiring significantly less training time and memory, with human and robustness analyses supporting the distinct advantages of fractal features. The findings highlight the potential of fractal-based representations for efficient, robust classification where substructure is crucial, and point to promising ensemble or architectural integration with DL systems.

Abstract

The fractal dimension provides a statistical index of object complexity by studying how the pattern changes with the measuring scale. Although useful in several classification tasks, the fractal dimension is under-explored in deep learning applications. In this work, we investigate the features that are learned by deep models and we study whether these deep networks are able to encode features as complex and high-level as the fractal dimensions. Specifically, we conduct a correlation analysis experiment to show that deep networks are not able to extract such a feature in none of their layers. We combine our analytical study with a human evaluation to investigate the differences between deep learning networks and models that operate on the fractal feature solely. Moreover, we show the effectiveness of fractal features in applications where the object structure is crucial for the classification task. We empirically show that training a shallow network on fractal features achieves performance comparable, even superior in specific cases, to that of deep networks trained on raw data while requiring less computational resources. Fractals improved the accuracy of the classification by 30% on average while requiring up to 84% less time to train. We couple our empirical study with a complexity analysis of the computational cost of extracting the proposed fractal features, and we study its limitation.

Figures (8)

• Figure 1: Computing the fractal features of an $M\times M$ image
• Figure 2: Samples from the datasets considered in this work. For binary classification datasets, we illustrate one random image from each category. For multi-class classification datasets, we choose two categories at random and we visualize one random image from each. (The images here are resized for display purposes)
• Figure 3: CKA and CCA scores on different layers in the DL models show consistently low correlation with the fractal features in comparison to other low-level features suggesting that such networks are unable to encode the fractal features.
• Figure 4: Maximum CKA and CCA correlation score between every pair of features averaged across all layers on DenseNet201 and the steel defect detection data
• Figure 5: Percentage of agreement on new testing images between ZFrac+NN and the DL models