Explorations in Texture Learning

TL;DR

This work investigates how convolutional neural networks learn textures and how texture cues influence object recognition by constructing texture–object associations using textures from the Describable Textures Dataset as input to a model pretrained on ImageNet. It introduces an effect-size metric to quantify how often texture images are classified as each object class, enabling a mapping from 47 texture classes to 1000 object classes. The results reveal three patterns: strong, expected texture–object mappings (e.g., honeycombed textures with honeycomb objects); strong but unexpected mappings (e.g., polka-dotted textures biasing bib predictions due to data); and expected but not present mappings (e.g., scaly textures not aligning with fish or reptiles). These findings advance interpretability and bias diagnosis in CNNs, highlighting how training data distributions shape learned texture representations and impact generalization.

Abstract

In this work, we investigate \textit{texture learning}: the identification of textures learned by object classification models, and the extent to which they rely on these textures. We build texture-object associations that uncover new insights about the relationships between texture and object classes in CNNs and find three classes of results: associations that are strong and expected, strong and not expected, and expected but not present. Our analysis demonstrates that investigations in texture learning enable new methods for interpretability and have the potential to uncover unexpected biases.

Figures (1)

• Figure 1: Examples of bibs with polka-dots in the ImageNet training data.