CLIDD: Cross-Layer Independent Deformable Description for Efficient and Discriminative Local Feature Representation

TL;DR

This work addresses the need for local feature descriptors that are both highly discriminative and computationally efficient for real-time spatial tasks. It introduces Cross-Layer Independent Deformable Description (CLIDD), which samples from multiple independent feature layers using learnable offsets, thereby avoiding costly dense feature maps. A hardware-aware kernel fusion strategy and a scalable training framework combining metric learning and knowledge distillation yield a family of model variants that achieve state-of-the-art accuracy with exceptional throughput on edge devices. The results demonstrate robust performance across homography, relative pose, and visual localization benchmarks, with ultra-compact models rivaling or surpassing much larger methods while maintaining real-time speeds. Overall, CLIDD provides a robust, scalable solution for real-time spatial intelligence tasks with minimal computational overhead.

Abstract

Robust local feature representations are essential for spatial intelligence tasks such as robot navigation and augmented reality. Establishing reliable correspondences requires descriptors that provide both high discriminative power and computational efficiency. To address this, we introduce Cross-Layer Independent Deformable Description (CLIDD), a method that achieves superior distinctiveness by sampling directly from independent feature hierarchies. This approach utilizes learnable offsets to capture fine-grained structural details across scales while bypassing the computational burden of unified dense representations. To ensure real-time performance, we implement a hardware-aware kernel fusion strategy that maximizes inference throughput. Furthermore, we develop a scalable framework that integrates lightweight architectures with a training protocol leveraging both metric learning and knowledge distillation. This scheme generates a wide spectrum of model variants optimized for diverse deployment constraints. Extensive evaluations demonstrate that our approach achieves superior matching accuracy and exceptional computational efficiency simultaneously. Specifically, the ultra-compact variant matches the precision of SuperPoint while utilizing only 0.004M parameters, achieving a 99.7% reduction in model size. Furthermore, our high-performance configuration outperforms all current state-of-the-art methods, including high-capacity DINOv2-based frameworks, while exceeding 200 FPS on edge devices. These results demonstrate that CLIDD delivers high-precision local feature matching with minimal computational overhead, providing a robust and scalable solution for real-time spatial intelligence tasks.

Figures (6)

• Figure 1: Structural comparison of local descriptor extraction strategies. (a) Sub-Scale Vanilla aggregates multi-scale features into a single low-resolution dense map for sparse sampling. (b) Full-Scale Vanilla constructs a high-resolution, unified feature field to improve precision at the cost of increased memory overhead. (c) Full-Scale SDDH introduces a sparse deformable sampling head on top of the dense, full-scale feature map. (d) Our method employs Cross-Layer Independent Deformable Description (CLIDD) to perform sparse sampling directly from multiple independent feature layers. This strategy generates highly discriminative descriptors while completely bypassing the construction of unified, dense feature maps.
• Figure 2: Precision-efficiency comparison on resource-constrained devices. This plot illustrates the relationship between pose estimation accuracy (AUC@10$^\circ$) on the MegaDepth-1500 benchmark and inference speed (FPS) measured on an edge platform. Our CLIDD-based models consistently occupy the upper-right quadrant, outperforming existing state-of-the-art methods by delivering higher matching precision and significantly faster processing speeds across all configurations.
• Figure 3: Mechanism of Cross-Layer Independent Deformable Description. This framework facilitates sparse feature extraction across multiple hierarchies without relying on a unified dense feature map. The Cross-Layer Predictor utilizes spatially aligned embeddings to determine level-specific sampling offsets for each feature resolution. Guided by these offsets, the Layer-Independent Sampler performs decoupled feature retrieval from each layer, where $M$ denotes the number of deformable sampling points per scale. This mechanism allows the model to capture both fine-grained geometry and semantic context simultaneously. The independently sampled features are then aggregated to produce a distinctive descriptor. While illustrated with three scales, this design is inherently scalable and generalizes to standard deformable sampling in single-layer configurations.
• Figure 4: Model Architecture. Our lightweight framework generates multi-scale representations using standard convolutional layers and ResNet blocks. The notation $C_i$ for $i \in \{1,2,3\}$ specifies the output channel counts for each feature block, while $C_{\mathrm{det}}$ and $C_{\mathrm{desc}}$ denote the dimensions for keypoint detection and final description. Within the description head, the total dimension of concatenated sampled features is defined as $C_{\mathrm{sum}} = C_1 + C_2 + C_3$. Symbols $\textcircled{\hbox{+}}$ and $\textcircled{\hbox{C}}$ indicate element-wise addition and concatenation operations, respectively. The term $r_i$ for $i \in \{2,3\}$ determines the number of ResNet blocks within each specific stage. Following extraction, a post-processing module transforms the dense detection heatmap into sparse keypoints. By utilizing our Cross-Layer Independent Deformable Description method, the architecture produces high-quality descriptors without generating dense, high-resolution feature maps, thereby simultaneously enhancing both descriptive precision and computational efficiency.
• Figure 5: Qualitative results on MegaDepth-1500. We compare ALIKED-N32, AWDesc-CA and DeDoDe-G against our U128 model.