Multi-Label Logo Recognition and Retrieval based on Weighted Fusion of Neural Features

TL;DR

The paper tackles multi-label logo recognition and retrieval by deploying a suite of specialized CNNs trained on Vienna-classified attributes (color, shape, semantics, and text) and fusing their neural codes through a weighted distance for flexible similarity search. It introduces a preprocessing pipeline that removes uniform backgrounds and text, enabling improved shape analysis, and compares several multi-label similarity strategies, with LabelPowerset and weighted fusion delivering strong results. Evaluated on a large EUIPO-derived 76k-logo dataset and benchmarked against METU, the approach yields substantial improvements in LRAP and NAR, and a qualitative survey indicates labeling performance can surpass that of human experts for figurative elements. The method provides a practical, automated aid for labeling, plagiarism detection, and trademark screening, enabling user-directed retrieval across multiple logo attributes while addressing the subjectivity inherent in semantic interpretation.

Abstract

Classifying logo images is a challenging task as they contain elements such as text or shapes that can represent anything from known objects to abstract shapes. While the current state of the art for logo classification addresses the problem as a multi-class task focusing on a single characteristic, logos can have several simultaneous labels, such as different colors. This work proposes a method that allows visually similar logos to be classified and searched from a set of data according to their shape, color, commercial sector, semantics, general characteristics, or a combination of features selected by the user. Unlike previous approaches, the proposal employs a series of multi-label deep neural networks specialized in specific attributes and combines the obtained features to perform the similarity search. To delve into the classification system, different existing logo topologies are compared and some of their problems are analyzed, such as the incomplete labeling that trademark registration databases usually contain. The proposal is evaluated considering 76,000 logos (7 times more than previous approaches) from the European Union Trademarks dataset, which is organized hierarchically using the Vienna ontology. Overall, experimentation attains reliable quantitative and qualitative results, reducing the normalized average rank error of the state-of-the-art from 0.040 to 0.018 for the Trademark Image Retrieval task. Finally, given that the semantics of logos can often be subjective, graphic design students and professionals were surveyed. Results show that the proposed methodology provides better labeling than a human expert operator, improving the label ranking average precision from 0.53 to 0.68.

Figures (11)

• Figure 1: Overview of the proposed method.
• Figure 2: Scheme of the proposed method.
• Figure 3: Examples of how selected text is removed from the image using CRAFT and how an inpainting neural network fills gaps.
• Figure 4: Schemes of the specialized CNNs (top, used in the MLC stage) and the Auto-Encoder (bottom, used for the similarity search). In this figure, the layer type is labeled with colors according to the side legend. Each layer configuration is shown in the scheme, including the activation function, the number of filters ($f$) and kernel size ($k$) for convolutions and transposed convolutions, the pool size ($p$) for max-pooling, the ratio $d$ used for dropout, the stride $st$ applied to each layer of the auto-encoder, and the number of neurons $n$ used for the fully-connected layers.
• Figure 5: Some examples of trademarks in the EUTM dataset. Note that some of them have only partial labeling of some characteristics, such as color and shape, and that the text is not labeled although it is present, as it is not considered to be a characteristic element of the design.