Multi-stage Bridge Inspection System: Integrating Foundation Models with Location Anonymization

TL;DR

This work tackles the dual challenge of accurate bridge damage assessment and regional privacy protection in field imagery. It introduces SAM3-based damage detection integrated with DBSCAN gap completion and a privacy framework that applies Gaussian blur to construction signs, complemented by OCR preprocessing to boost text recognition, all accelerated on GPUs to achieve around 1.7 seconds per image. The approach achieves high detection performance (e.g., $\text{precision}_{crack}=0.942$, $\text{recall}_{crack}=0.918$, $\text{precision}_{rebar}=0.961$, $\text{recall}_{rebar}=0.935$, $F1\approx0.951$) and strong privacy protection (e.g., $97.3\%$ sign detection, $99.1\%$ regional coverage, $98.7\%$ utility retention) with a $51 \times 51$ blur kernel and $k \ge 3$ regional anonymity. The system is open-source and designed for real-world deployment, balancing safety, inspection utility, and user privacy, and offers a practical, scalable solution for privacy-conscious infrastructure monitoring in Japan and beyond.

Abstract

In Japan, civil infrastructure condition monitoring is mandated through visual inspection every five years. Field-captured damage images frequently contain concrete cracks and rebar exposure, often accompanied by construction signs revealing regional information. To enable safe infrastructure use without causing public anxiety, it is essential to protect regional information while accurately extracting damage features and visualizing key indicators for repair decision-making. This paper presents an open-source bridge damage detection system with regional privacy protection capabilities. We employ Segment Anything Model (SAM) 3 for rebar corrosion detection and utilize DBSCAN for automatic completion of missed regions. Construction sign regions are detected and protected through Gaussian blur. Four preprocessing methods improve OCR accuracy, and GPU optimization enables 1.7-second processing per image. The technology stack includes SAM3, PyTorch, OpenCV, pytesseract, and scikit-learn, achieving efficient bridge inspection with regional information protection.

Figures (5)

• Figure 1: Overview of SAM3-based Inspection Pipeline with Location Anonymization. The comprehensive workflow integrates multi-stage damage detection with privacy protection mechanisms, ensuring accurate structural assessment while protecting sensitive regional information through automated anonymization techniques. (Created by NotebookLM)
• Figure 2: SAM3 Damage Detection Pipeline - Phase 1: Initial detection and pattern recognition. The flowchart illustrates the first phase: (1) Image input and SAM ViT-H model loading (FP32, 2.4GB GPU VRAM), (2) Method 1 auto-detection with 1024-point grid sampling, (3) HSV color space analysis for rust detection (H:0-177, S:31-135, V:28-142), (4) Method 2 precise detection using dense grid prompts (5$\times$5/7$\times$7/9$\times$9), (5) SAM inference for mask generation with shape filtering (aspect ratio $\geq$2.0, area 70-2000px), (6) Rebar pattern analysis using DBSCAN and least squares algorithms, and (7) Critical decision point determining whether 2+ parallel lines are detected, leading to different processing paths in Figure \ref{['fig:sam3_pipeline_part2']}.
• Figure 3: Phase 2: Pattern prediction and privacy protection. The flowchart illustrates the second phase with two input paths from Figure \ref{['fig:sam3_pipeline_part1']}: (1) Pattern-detected path leading to Method 3 pattern prediction with additional prompt generation (on-line, between, outside positions), followed by SAM inference for detecting additional regions and deduplication (50%+ overlap removal), (2) No-pattern path proceeding directly to privacy protection, (3) Privacy protection phase including automatic signboard detection using white HSV color space and SAM, (4) Gaussian blur application (51$\times$51 masking) for regional information protection, (5) Final visualization with 12-color coding, and (6) Comprehensive result output including combined images, privacy-protected versions, and JSON statistics.
• Figure 4: Damage detection mask results showing detected corrosion regions with privacy protection applied. The system successfully identifies multiple corrosion areas while maintaining regional anonymity through automated masking techniques.
• Figure 5: Comprehensive corrosion detection results showing original image, detected corrosion regions, and segmentation masks. The multi-stage detection approach successfully identifies various scales of rebar corrosion from microscopic rust initiation to extensive deterioration patterns.