An Analysis of Multi-Task Architectures for the Hierarchic Multi-Label Problem of Vehicle Model and Make Classification

TL;DR

This work analyzes the advantages and limitations of multi-task learning in a hierarchical multi-label classification problem: car make and model classification and yields significant improvements on the CompCars dataset for both types of models.

Abstract

Most information in our world is organized hierarchically; however, many Deep Learning approaches do not leverage this semantically rich structure. Research suggests that human learning benefits from exploiting the hierarchical structure of information, and intelligent models could similarly take advantage of this through multi-task learning. In this work, we analyze the advantages and limitations of multi-task learning in a hierarchical multi-label classification problem: car make and model classification. Considering both parallel and cascaded multi-task architectures, we evaluate their impact on different Deep Learning classifiers (CNNs, Transformers) while varying key factors such as dropout rate and loss weighting to gain deeper insight into the effectiveness of this approach. The tests are conducted on two established benchmarks: StanfordCars and CompCars. We observe the effectiveness of the multi-task paradigm on both datasets, improving the performance of the investigated CNN in almost all scenarios. Furthermore, the approach yields significant improvements on the CompCars dataset for both types of models.

Figures (8)

• Figure 1: Simplified overview of multi-task architectures. (A) Parallel (B) Cascaded (C) Cross-Talk Multitask.
• Figure 2: Test accuracy for car model prediction for the initial experiment used for choosing the most promising base models on Stanford Cars (no dropout, $\lambda_1 = 0.9$, $\lambda_2=0.1$).
• Figure 3: Test accuracy for car model prediction for the initial experiment used for choosing the most promising base models on CompCars (no dropout, $\lambda_1 = 0.9$, $\lambda_2=0.1$).
• Figure 4: Performance on CompCars test set for make prediction depending on MTL model and architecture choice when dropout is set to 0.5. For each pair of lambda values and each investigated model we present three bars describing the performance of each combination of tasks: ST, parallel MTL and cascaded MTL.
• Figure 5: Performance on CompCars test set depending on MTL model and architecture choice when dropout is set to 0.25.A set of three bars is showcased for the investigation combinations of models and lambda values.The first bar describes the performance of the ST model, while the second and third show the accuracy of the parallel and cascaded MTL variants.