Multi-label Classification using Deep Multi-order Context-aware Kernel Networks

TL;DR

This paper tackles multi-label image classification by exploiting rich contextual geometry through a learnable context-aware kernel framework. It introduces the Deep Multi-order Context-aware Kernel Network (DMCKN), which unfolds kernel updates into a deep network that aggregates content and multi-order spatial context via random walks and attention, producing explicit kernel mappings for each image. The approach is trained end-to-end with a grouped loss to handle label imbalance, and includes architectural elements like 1×1 convolutions for dimensionality control. Empirical results on Corel5K and NUS-WIDE demonstrate that DMCKN consistently outperforms baselines and benefits from higher-order context and effective random-walk strategies, suggesting strong practical impact for context-aware visual recognition.

Abstract

Multi-label classification is a challenging task in pattern recognition. Many deep learning methods have been proposed and largely enhanced classification performance. However, most of the existing sophisticated methods ignore context in the models' learning process. Since context may provide additional cues to the learned models, it may significantly boost classification performances. In this work, we make full use of context information (namely geometrical structure of images) in order to learn better context-aware similarities (a.k.a. kernels) between images. We reformulate context-aware kernel design as a feed-forward network that outputs explicit kernel mapping features. Our obtained context-aware kernel network further leverages multiple orders of patch neighbors within different distances, resulting into a more discriminating Deep Multi-order Context-aware Kernel Network (DMCKN) for multi-label classification. We evaluate the proposed method on the challenging Corel5K and NUS-WIDE benchmarks, and empirical results show that our method obtains competitive performances against the related state-of-the-art, and both quantitative and qualitative performances corroborate its effectiveness and superiority for multi-label image classification.

Figures (5)

• Figure 1: Deep Multi-order Context-aware Kernel Network framework.
• Figure 2: Multi-order neighborhood system. The left side shows the first-order and second-order neighborhoods. On the right, the third-order neighborhood is built from the second-order neighborhood based on the transition probabilities.
• Figure 3: Details of the Deep Multi-order Context-aware Kernel Network. "RWCA" is the abbreviation of Random Walk and Context Awareness.
• Figure 4: Image instances of the initial and learned context of higher-order domains on the Corel5K dataset (upper half) and the NUS-WIDE dataset (lower half). From the left to right column: the original images, the initial multi-order neighborhood system, the learned different levels of neighborhoods on the central cell, the impacts of different cells. Warmer color stands for higher impact.
• Figure 5: Comparison of image instances of predicted labels and actual labels including FC (First-Order Context), SC (Second-Order Context), and TC (Third-Order Context), the left two images are from the Corel5K dataset and the right two images are from the NUS-WIDE dataset.