Ranking-based Adaptive Query Generation for DETRs in Crowded Pedestrian Detection

TL;DR

The paper addresses the sensitivity of DETR-based detectors to a fixed query count $K$ in crowded pedestrian detection. It introduces Rank-based Adaptive Query Generation (RAQG), comprising a ranking prediction head to estimate the required query count $R$, a query supplementer to balance positive/negative samples, and Soft Gradient L1 loss to train the ranking head; queries are generated from the top-$R$ detections. RAQG is shown to be universal across DETR variants and achieves competitive results on Crowdhuman and Citypersons, including a state-of-the-art MR on Crowdhuman with negligible overhead. By removing manual hyper-parameter tuning of query numbers, RAQG improves robustness in crowded scenes and offers a plug-in solution for DETR-based pedestrian detection.

Abstract

DEtection TRansformer (DETR) and its variants (DETRs) have been successfully applied to crowded pedestrian detection, which achieved promising performance. However, we find that, in different degrees of crowded scenes, the number of DETRs' queries must be adjusted manually, otherwise, the performance would degrade to varying degrees. In this paper, we first analyze the two current query generation methods and summarize four guidelines for designing the adaptive query generation method. Then, we propose Rank-based Adaptive Query Generation (RAQG) to alleviate the problem. Specifically, we design a rank prediction head that can predict the rank of the lowest confidence positive training sample produced by the encoder. Based on the predicted rank, we design an adaptive selection method that can adaptively select coarse detection results produced by the encoder to generate queries. Moreover, to train the rank prediction head better, we propose Soft Gradient L1 Loss. The gradient of Soft Gradient L1 Loss is continuous, which can describe the relationship between the loss value and the updated value of model parameters granularly. Our method is simple and effective, which can be plugged into any DETRs to make it query-adaptive in theory. The experimental results on Crowdhuman dataset and Citypersons dataset show that our method can adaptively generate queries for DETRs and achieve competitive results. Especially, our method achieves state-of-the-art 39.4% MR on Crowdhuman dataset.

Figures (6)

• Figure 1: Framework comparison of the different query generation methods. The icons with brown background are the parts of the query generation methods. Since the backbone is unrelated to query generation, we omitted it in the figure. (a) Two-stage treats the transformer encoder as a region proposal network to produce coarse and dense detection results. Subsequently, the K highest score detection results are selected for generating K queries. (b) Learnable Parameters utilizes K trainable embeddings to generate K queries. (c) Our RAQG first predicts the number of queries, X, and selects X highest score detection results to generate X queries. The K is a hyper-parameter in (a)(b) that needs to be adjusted manually. The X is predicted in (c).
• Figure 2: The visualization of the different number of queries generated by Learnable Parameters.
• Figure 3: The visualization of the different number of queries generated by Two-stage.
• Figure 4: The pipeline of DETRs equipped with our RAQG. The part with red background is RAQG. The part with yellow or blue background is the common part of DETRs.
• Figure 5: The network of our ranking prediction head.