Differentiable NMS via Sinkhorn Matching for End-to-End Fabric Defect Detection

TL;DR

This work tackles fabric defect detection by removing the non-differentiable bottleneck of traditional NMS and enabling end-to-end optimization. It introduces differentiable NMS via Sinkhorn-Knopp as a differentiable bipartite matching between proposals and latent defect regions, complemented by an entropy-constrained mask refinement implemented with Frank–Wolfe and a Total Variation regularizer. The approach yields significant gains on the Tianchi fabric dataset (e.g., mAP improvements and real-time inference) while remaining architecture-agnostic and generalizable to general object detection, as evidenced by competitive COCO results. The framework provides theoretical convergence guarantees for both Sinkhorn matching and entropy refinement, supporting robust gradient flow and stable training for industrial deployment.

Abstract

Fabric defect detection confronts two fundamental challenges. First, conventional non-maximum suppression disrupts gradient flow, which hinders genuine end-to-end learning. Second, acquiring pixel-level annotations at industrial scale is prohibitively costly. Addressing these limitations, we propose a differentiable NMS framework for fabric defect detection that achieves superior localization precision through end-to-end optimization. We reformulate NMS as a differentiable bipartite matching problem solved through the Sinkhorn-Knopp algorithm, maintaining uninterrupted gradient flow throughout the network. This approach specifically targets the irregular morphologies and ambiguous boundaries of fabric defects by integrating proposal quality, feature similarity, and spatial relationships. Our entropy-constrained mask refinement mechanism further enhances localization precision through principled uncertainty modeling. Extensive experiments on the Tianchi fabric defect dataset demonstrate significant performance improvements over existing methods while maintaining real-time speeds suitable for industrial deployment. The framework exhibits remarkable adaptability across different architectures and generalizes effectively to general object detection tasks.

Figures (9)

• Figure 1: Textile manufacturing system with automated inspection zones (highlighted in red) and sample fabric outputs showing pattern variations relevant for defect detection.
• Figure 2: End-to-end fabric defect detection architecture. Top: Complete pipeline showing fabric image input through proposal generation, differentiable NMS via Hungarian matching, and entropy-constrained refinement. Bottom: Detailed network components including ResNet backbone, Feature Pyramid Network with lateral connections, and Region Proposal Network with scale-specific anchors.
• Figure 3: Detailed illustration of the differentiable NMS via Hungarian matching. Multiple proposal regions on a fabric image (left) are matched to latent defect regions through a differentiable assignment process (right). The cost matrix represents assignment costs between proposals and latent regions, with lower values (in red) indicating preferred assignments. The Sinkhorn-Knopp algorithm transforms this into a differentiable soft assignment matrix where higher values (in green) indicate stronger assignments.
• Figure 4: Statistical analysis of the Tianchi textile defect dataset: (a) Category-wise distribution of images and annotations; (b) Spatial annotation density visualization with bubble diameter proportional to annotations-per-image ratio.
• Figure 5: Performance comparison of fabric defect detection methods across different defect categories. The radar charts illustrate mAP (%) values for (a) structure-related defects including broken holes, water stains, and thread-related abnormalities, and (b) surface anomalies such as star jumps, repair marks, and rolling marks.