TrajGATFormer: A Graph-Based Transformer Approach for Worker and Obstacle Trajectory Prediction in Off-site Construction Environments

TL;DR

This work tackles safe trajectory prediction in off-site construction by integrating robust object detection and tracking (YOLOv10n and DeepSORT) with transformer-based, graph-aware trajectory predictors (TrajGATFormer and TrajGATFormer-Obstacle). By modeling temporal dynamics and social interactions among workers and moving obstacles via Graph Attention Networks within a Transformer encoder-decoder, the approach achieves notable improvements over baseline methods on construction-site data. The authors also employ a transfer-learning strategy using ETH data to initialize the models before fine-tuning on construction data, and they report substantial ADE/FDE reductions (up to around 40% in some cases) and a clear advantage when obstacles are modeled. The work demonstrates practical potential for real-time collision-avoidance in dynamic construction environments and outlines future directions toward larger, more diverse datasets and live-site integration.

Abstract

As the demand grows within the construction industry for processes that are not only faster but also safer and more efficient, offsite construction has emerged as a solution, though it brings new safety risks due to the close interaction between workers, machinery, and moving obstacles. Predicting the future trajectories of workers and taking into account social and environmental factors is a crucial step for developing collision-avoidance systems to mitigate such risks. Traditional methods often struggle to adapt to the dynamic and unpredictable nature of construction environments. Many rely on simplified assumptions or require hand-crafted features, limiting their ability to respond to complex, real-time interactions between workers and moving obstacles. While recent data-driven methods have improved the modeling of temporal patterns, they still face challenges in capturing long-term behavior and accounting for the spatial and social context crucial to collision risk assessment. To address these limitations, this paper proposes a framework integrating YOLOv10n and DeepSORT for precise detection and tracking, along with two novel trajectory prediction models: TrajGATFormer and TrajGATFormer-Obstacle. YOLOv10n serves as the backbone for object detection, accurately identifying workers and obstacles in diverse scenes, while DeepSORT efficiently tracks them over time with unique IDs for continuity. Both models employ a transformer encoder-decoder with Graph Attention Networks (GAT) to capture temporal and spatial interactions. TrajGATFormer predicts worker trajectories with an ADE of 1.25 m and FDE of 2.3 m over a 4.8 s horizon, while TrajGATFormer-Obstacle extends prediction to both workers and obstacles, achieving higher accuracy (ADE 1.15 m, FDE 2.2 m). Comparative analysis shows both models outperform traditional methods, reducing ADE and FDE by up to 35% and 38%, respectively.

Figures (8)

• Figure 1: Overview of the proposed framework.
• Figure 2: Overview of our TrajGATFormer worker trajectory prediction model.
• Figure 3: Overview of our TrajGATFormer-Obstacle worker and obstacle trajectory prediction model.
• Figure 4: Statistical analysis of the construction dataset.
• Figure 5: Training loss curves comparison between training with transfer learning and without for TrajGATFormer.