DetailSemNet: Elevating Signature Verification through Detail-Semantic Integration

TL;DR

DetailSemNet addresses offline signature verification by prioritizing local patch-level structure and preserving high-frequency details often lost in transformer backbones. It introduces a Detail-Semantics Integrator to split features into semantic and detail components and a Structural Matching module to align local tokens, trained with a double-margin contrastive loss. The approach achieves state-of-the-art results on BHSig-B/H and CEDAR datasets and demonstrates strong cross-dataset generalization and interpretability through visualized patch matching. This work highlights the value of combining local patch matching with semantic-aware feature integration for robust, real-world OSV applications.

Abstract

Offline signature verification (OSV) is a frequently utilized technology in forensics. This paper proposes a new model, DetailSemNet, for OSV. Unlike previous methods that rely on holistic features for pair comparisons, our approach underscores the significance of fine-grained differences for robust OSV. We propose to match local structures between two signature images, significantly boosting verification accuracy. Furthermore, we observe that without specific architectural modifications, transformer-based backbones might naturally obscure local details, adversely impacting OSV performance. To address this, we introduce a Detail Semantics Integrator, leveraging feature disentanglement and re-entanglement. This integrator is specifically designed to enhance intricate details while simultaneously expanding discriminative semantics, thereby augmenting the efficacy of local structural matching. We evaluate our method against leading benchmarks in offline signature verification. Our model consistently outperforms recent methods, achieving state-of-the-art results with clear margins. The emphasis on local structure matching not only improves performance but also enhances the model's interpretability, supporting our findings. Additionally, our model demonstrates remarkable generalization capabilities in cross-dataset testing scenarios. The combination of generalizability and interpretability significantly bolsters the potential of DetailSemNet for real-world applications.

Figures (7)

• Figure 1: Three samples from the ChiSig dataset. Signature (a) originates from a different individual than signatures (b) and (c). At first glance, these signatures appear remarkably similar when viewed holistically. However, detailed analysis at the patch level reveals distinct differences between them, which are aspects frequently overlooked in previous methodologies.
• Figure 2: We employ filters to extract Low-frequency (LF), low-plus-middle frequency (LMF), and low-plus-high frequency (LHF) images. Our model captures both semantic pattern (low-frequency) and stroke structure and style detail (high-frequency) for improved verification. Leveraging high-frequency data enhances performance, unlike the baseline transformer model, which solely relies on low-frequency patterns and does not benefit from high-frequency features.
• Figure 3: Conventional OSV method vs. Our proposed method: The left figure shows the traditional approach, lacking detailed feature information and relying solely on global similarity for comparison. On the right, our method, called DetailSemNet, employs the Detail-Semantics Integrator to divide features into Semantic and Detail components. The Semantic component acquires contextual information through the SemanticsAttend Branch, while the Detail component is processed via the SalientConv and DetailConv Branches. Integrating these outputs yields feature representations containing both detailed and semantic information. Additionally, the model utilizes Structural Matching techniques to emphasize detailed information alongside global similarity.
• Figure 4: In our DetailSemNet, the Detail-Semantics Integrator splits features into two components: Semantic and Detail. The Semantic component is sent to the SemanticsAttend Branch to gather context information, while the Detail component goes through the SalientConv Branch and DetailConv Branch. The outputs of these three branches are then fused, creating features that incorporate both detailed and semantic information. In addition to global similarity, the model also performs structural matching on the features, allowing it to pay more attention to detailed information.
• Figure 5: Illustrating the matching results of our model on signature pairs. The sample pairs are selected from the four datasets. Our model demonstrates correct matching results when tested on positive pairs; whereas, when tested on negative pairs, it exhibits matching at incorrect positions, sometimes even at multiple locations.