A Semantic-Enhanced Heterogeneous Graph Learning Method for Flexible Objects Recognition

TL;DR

The paper tackles the challenge of recognizing flexible objects whose shapes, translucency, and inter-class differences complicate traditional recognition paradigms. It introduces a semantic-enhanced heterogeneous graph learning framework that fuses global visual cues and local semantic information through an adaptive scanning module and a heterogeneous graph generator, enabling better cross-modal alignment. The method is validated on FDA, FSCW, CIFAR-100, and ImageNet-Hard, achieving competitive or superior accuracies (e.g., 80.50% on FDA and 89.98% on FSCW) and demonstrating robustness across hyperparameters and ablations. This approach advances flexible object recognition by integrating state-space-inspired semantic extraction with dynamic graph-based reasoning, offering improved generalization and practical impact for complex visual understanding.

Abstract

Flexible objects recognition remains a significant challenge due to its inherently diverse shapes and sizes, translucent attributes, and subtle inter-class differences. Graph-based models, such as graph convolution networks and graph vision models, are promising in flexible objects recognition due to their ability of capturing variable relations within the flexible objects. These methods, however, often focus on global visual relationships or fail to align semantic and visual information. To alleviate these limitations, we propose a semantic-enhanced heterogeneous graph learning method. First, an adaptive scanning module is employed to extract discriminative semantic context, facilitating the matching of flexible objects with varying shapes and sizes while aligning semantic and visual nodes to enhance cross-modal feature correlation. Second, a heterogeneous graph generation module aggregates global visual and local semantic node features, improving the recognition of flexible objects. Additionally, We introduce the FSCW, a large-scale flexible dataset curated from existing sources. We validate our method through extensive experiments on flexible datasets (FDA and FSCW), and challenge benchmarks (CIFAR-100 and ImageNet-Hard), demonstrating competitive performance.

Figures (4)

• Figure 1: Raster Scan suffer from discontinuities, while Local Scan struggles with semantic irrelevance. In contrast, our adaptive scanning method enables the state space model to better capture semantic node context, extracting more discriminative features for flexible objects of varying shapes and sizes.
• Figure 2: Method overview. Semantic-enhanced heterogeneous graph learning that employs an adaptive scanning module, enables extraction of discriminative semantic context for matching flexible objects with varying shapes and sizes, while aligning semantic and visual nodes to enhance cross-modal feature correlation. Additionally, heterogeneous graph generation allows the graph neural network to effectively aggregate global visual and local semantic node features.
• Figure 3: (a) Patch rearrangement results based on different scanning paths. The local scan method causes semantic irrelevance, while our adaptive scan method effectively captures discriminative semantic contexts. (b) Aggregation results of the central node (red, representing semantic information) and neighboring nodes (blue, representing visual information). Our method outperforms the ViG method in distinguishing flexible objects from the background.
• Figure 4: Influence of hyperparameters on performance.