Segmentation and Characterization of Macerated Fibers and Vessels Using Deep Learning

TL;DR

This study tackles the challenge of segmenting and characterizing overlapping, translucent wood cells in macerated leaf and stem images. It introduces a one-stage deep learning pipeline based on YOLOv8-seg trained on a large, annotated wood macerate dataset to detect, segment, and morphometrically quantify fibers and vessels, achieving $mAP_{0.5-0.95}=0.78$ on large images. The approach is robust across staining variations, provides per-object length, width, and area measurements, and is complemented by a user-friendly GUI and Colab-based training resources. Validation on a GA20ox1 transgenic line demonstrates biologically meaningful differences in fiber length, underscoring the method’s practical utility for high-throughput wood cell analysis and genotype-phenotype studies.

Abstract

Wood comprises different cell types, such as fibers, tracheids and vessels, defining its properties. Studying cells' shape, size, and arrangement in microscopy images is crucial for understanding wood characteristics. Typically, this involves macerating (soaking) samples in a solution to separate cells, then spreading them on slides for imaging with a microscope that covers a wide area, capturing thousands of cells. However, these cells often cluster and overlap in images, making the segmentation difficult and time-consuming using standard image-processing methods. In this work, we developed an automatic deep learning segmentation approach that utilizes the one-stage YOLOv8 model for fast and accurate segmentation and characterization of macerated fiber and vessel form aspen trees in microscopy images. The model can analyze 32,640 x 25,920 pixels images and demonstrate effective cell detection and segmentation, achieving a mAP_{0.5-0.95} of 78 %. To assess the model's robustness, we examined fibers from a genetically modified tree line known for longer fibers. The outcomes were comparable to previous manual measurements. Additionally, we created a user-friendly web application for image analysis and provided the code for use on Google Colab. By leveraging YOLOv8's advances, this work provides a deep learning solution to enable efficient quantification and analysis of wood cells suitable for practical applications.

Figures (7)

• Figure 1: Schematic of the YOLOv8 architecture for fiber and vessel segmentation. The model contains a Feature Extractor for feature extraction, Feature Fusion for feature aggregation, Prediction Head for predicting the objects' bounding boxes, classes, and masks. The loss component is used to optimize the model performance. An input image is passed through the network, which performs classification, detection, and segmentation jointly. This enables the delineation of individual cells even when overlapping, as shown in the prediction output.
• Figure 2: Example of a segmented object and the corresponding image analysis that automatically measures the object's morphological traits using our YOLOv8 model. Panel (a) displays the original microscopy input image used for object detection, which is sent to the YOLOv8 model (b) that outputs detected objects. Panel (c) shows the individual mask generated for each individual cell in the image and also demonstrates that full masks are obtained from translucent overlapping objects. Panel (d) illustrates the measurements of length and width for each detected object, providing quantitative data extracted from the image.
• Figure 3: These plots showcase the precision-recall (a, c) and F1-confidence (b, d) curves used to evaluate the performance of YOLOv8x in detecting (a,b) and segmenting (c,d) fibers and vessels. The model demonstrates strong mAP and F1-score across thresholds, confirming its effectiveness in object detection and segmentation tasks.
• Figure 4: The images depict YOLOv8 effectively detecting and segmenting images obtained with various staining protocols and color acquisition parameters. Panels in (a) depict only Safranin stained fibers with diverse backgrounds typically associated with imperfect white balance and (b) shows from left to right, Toluidine blue stained samples, unstained, unstained with properly adjusted white balance, and unstained acquired in grayscale mode. Note that in these representations, the overlayed mask of fibers and vessels is displayed as red and should not be confused with red staining.
• Figure 5: The detection of fibers and vessels encountered some challenges in densely packed regions, as shown in these examples where several fibers and vessels were not properly detected or segmented.