Multimodal surface defect detection from wooden logs for sawing optimization

TL;DR

This work tackles knot detection on log surfaces to optimize sawing by introducing a multimodal framework that fuses RGB surface images with height maps derived from laser scans to infer knot locations and internal structure without relying on CT. It employs two parallel FPNSegmentation-based streams, one for stitched RGB maps and one for height maps, with a late fusion module that concatenates last-layer features to produce knot predictions, and it supports single-modality operation when needed. A 2D cross-correlation-based sawing-angle optimization uses a pattern function $f_p(\theta)$ and a knot function $f_k(\theta)$ to select rotations that minimize arris knots. On a dataset of nine Scots pine logs (with CT ground truth and pretraining data from additional laser logs), the fusion approach outperforms single-modality methods and reduces arris knots by 23% and their area by 31% compared with random sawing, demonstrating a practical CT-free pathway for sawmills.

Abstract

We propose a novel, good-quality, and less demanding method for detecting knots on the surface of wooden logs using multimodal data fusion. Knots are a primary factor affecting the quality of sawn timber, making their detection fundamental to any timber grading or cutting optimization system. While X-ray computed tomography provides accurate knot locations and internal structures, it is often too slow or expensive for practical use. An attractive alternative is to use fast and cost-effective log surface measurements, such as laser scanners or RGB cameras, to detect surface knots and estimate the internal structure of wood. However, due to the small size of knots and noise caused by factors, such as bark and other natural variations, detection accuracy often remains low when only one measurement modality is used. In this paper, we demonstrate that by using a data fusion pipeline consisting of separate streams for RGB and point cloud data, combined by a late fusion module, higher knot detection accuracy can be achieved compared to using either modality alone. We further propose a simple yet efficient sawing angle optimization method that utilizes surface knot detections and cross-correlation to minimize the amount of unwanted arris knots, demonstrating its benefits over randomized sawing angles.

Figures (13)

• Figure 1: Task illustration for image-based and point cloud-based methods. The top images show a sample of a log surface represented by an image (a) and a height map generated from point cloud (b). Middle images show the samples with X-ray-based annotations. The bottom row adds the model predictions based on only one modality to the samples.
• Figure 2: Illustration of different knot locations.
• Figure 3: Overall fusion architecture diagram.
• Figure 4: RGB image stitching diagram. Images are first cropped, adjusted, and then iteratively stitched forming the final stitched RGB surface image.
• Figure 5: Overall sawing optimization method pipeline.