m^3TrackFormer: Transformer-based mmWave Multi-Target Tracking with Lost Target Re-Acquisition Capability

TL;DR

This paper tackles robust tracking for mmWave ISAC systems with pencil-like beams that risk losing track of targets. It introduces a two-mode Transformer framework, m^2TrackFormer for a single target and m^3TrackFormer for multiple targets, featuring an N-Mode encoder-based network for normal tracking and an R-Mode decoder-based network for re-acquisition that leverages both historical trajectories and loss feedback. The approach achieves higher tracking duration, improved successful tracking probability, and millisecond-level latency even as the number of targets grows, by exploiting global trajectory features and a residual fusion strategy for lost-event information. This work offers a practical, scalable solution for real-time joint sensing and communication in 6G mmWave ISAC systems, with strong potential for rapid re-acquisition and reduced beam sweeping overhead.

Abstract

This paper considers a millimeter wave (mmWave) integrated sensing and communication (ISAC) system, where a base station (BS) equipped with a large number of antennas but a small number of radio-frequency (RF) chains emits pencillike narrow beams for persistent tracking of multiple moving targets. Under this model, the tracking lost issue arising from the misalignment between the pencil-like beams and the true target positions is inevitable, especially when the trajectories of the targets are complex, and the conventional Kalman filter-based scheme does not work well. To deal with this issue, we propose a Transformer-based mmWave multi-target tracking framework, namely m3TrackFormer, with a novel re-acquisition mechanism, such that even if the echo signals from some targets are too weak to extract sensing information, we are able to re-acquire their locations quickly with small beam sweeping overhead. Specifically, the proposed framework can operate in two modes of normal tracking and target re-acquisition during the tracking procedure, depending on whether the tracking lost occurs. When all targets are hit by the swept beams, the framework works in the Normal Tracking Mode (N-Mode) with a Transformer encoder-based Normal Tracking Network (N-Net) to accurately estimate the positions of these targets and predict the swept beams in the next time block. While the tracking lost happens, the framework will switch to the Re-Acquisition Mode (R-Mode) with a Transformer decoder-based Re-Acquisition Network (RNet) to adjust the beam sweeping strategy for getting back the lost targets and maintaining the tracking of the remaining targets. Thanks to the ability of global trajectory feature extraction, the m3TrackFormer can achieve high beam prediction accuracy and quickly re-acquire the lost targets, compared with other tracking methods.

Figures (10)

• Figure 1: System model for target tracking in the 6G mmWave ISAC system: tracking lost event occurs when the pencil-like mmWave beams are not precisely pointed to the target positions.
• Figure 2: The transmission protocol for the considered mmWave ISAC system.
• Figure 3: The proposed Transformer-based tracking framework in the single-target case, namely $\text{m}^2$TrackFormer. In the predictor $\tilde{\mathcal{F}}^{(q)}(\cdot)$, the left upper branch illustrates the N-Net, where the historical sensing information is processed sequentially by the MAFE and TSP modules to predict future beam directions. The left lower branch shows the R-Net, in which the historical sensing information and feedback from the tracking lost events are processed via the MAFE and LE modules, respectively, followed by the feature fusion in the FF module to adjust future beam sweeping strategy. The right branch shows the multi-head Masked Attention Mechanism employed in the Self-Attention layers and Cross-Attention layers.
• Figure 4: The diagram for the proposed $\text{m}^3$Trackformer.
• Figure 5: Tracking Performance of successful probability versus tracking time.

Theorems & Definitions (2)

• Remark 1
• Remark 2