TimberVision: A Multi-Task Dataset and Framework for Log-Component Segmentation and Tracking in Autonomous Forestry Operations

TL;DR

TimberVision tackles the challenge of automating forestry operations by providing a large RGB dataset with detailed trunk-component annotations (including live trees and cut logs) and a unified framework to fuse oriented object detection and instance segmentation outputs into robust trunk representations for real-time tracking. The authors perform extensive ablations to understand the impact of class definitions, model capacity, and scene parameters, and demonstrate a practical multi-task pipeline that achieves competitive detection, segmentation, and tracking performance while remaining RGB-only. A key contribution is the fusion framework that combines component-level detections into unified trunks and tracks them over time, enabling geometric inferences such as middle axes and boundaries. Cross-dataset fine-tuning shows strong generalization potential to related forestry datasets, and the work highlights the dataset’s value for safety-critical, autonomous forestry applications where depth data may be unavailable or costly.

Abstract

Timber represents an increasingly valuable and versatile resource. However, forestry operations such as harvesting, handling and measuring logs still require substantial human labor in remote environments posing significant safety risks. Progressively automating these tasks has the potential of increasing their efficiency as well as safety, but requires an accurate detection of individual logs as well as live trees and their context. Although initial approaches have been proposed for this challenging application domain, specialized data and algorithms are still too scarce to develop robust solutions. To mitigate this gap, we introduce the TimberVision dataset, consisting of more than 2k annotated RGB images containing a total of 51k trunk components including cut and lateral surfaces, thereby surpassing any existing dataset in this domain in terms of both quantity and detail by a large margin. Based on this data, we conduct a series of ablation experiments for oriented object detection and instance segmentation and evaluate the influence of multiple scene parameters on model performance. We introduce a generic framework to fuse the components detected by our models for both tasks into unified trunk representations. Furthermore, we automatically derive geometric properties and apply multi-object tracking to further enhance robustness. Our detection and tracking approach provides highly descriptive and accurate trunk representations solely from RGB image data, even under challenging environmental conditions. Our solution is suitable for a wide range of application scenarios and can be readily combined with other sensor modalities.

Figures (21)

• Figure 1: Representative examples of semi-automatically generated annotations for instance segmentation of multiple trunk components in the TimberVision dataset.
• Figure 2: Representative image samples for Low, Mid and High intensities of annotated scene parameters. The color bars show their distributions across the dataset.
• Figure 3: Overview of our annotation pipeline automatically deriving multi-task annotations from manual point annotations.
• Figure 4: Overview of our task-fusion algorithm deriving unified trunk representations from OOD and ISEG outputs for individual component classes. Processing steps are denoted by rounded and intermediate results by square boxes.
• Figure 5: Accuracy as mean class mAP50-95 and average inference time on test set for multiple model capacities and image sizes.